-- Logs begin at Thu 2022-07-14 10:26:12 UTC, end at Thu 2022-07-14 11:44:39 UTC. -- Jul 14 10:57:01.502367 dv-cinder-822821-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 10:57:01.510167 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 10:57:01 2022] *** Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: nodename: dv-cinder-822821-5-iscsi Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: machine: x86_64 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: clock source: unix Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: pcre jit disabled Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: detected number of CPU cores: 4 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: current working directory: / Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: detected binary path: /usr/bin/uwsgi-core Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: your processes number limit is 47680 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: your memory page size is 4096 bytes Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: detected max file descriptor number: 2048 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: lock engine: pthread robust mutexes Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: thunder lock: enabled Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 10:57:01.513575 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 10:57:01.558045 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: Python main interpreter initialized at 0x56217c3849c0 Jul 14 10:57:01.558045 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: python threads support enabled Jul 14 10:57:01.558045 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: your server socket listen backlog is limited to 100 connections Jul 14 10:57:01.558045 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: your mercy for graceful operations on workers is 90 seconds Jul 14 10:57:01.558445 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 10:57:01.558519 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: *** Operational MODE: preforking *** Jul 14 10:57:01.558780 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: *** uWSGI is running in multiple interpreter mode *** Jul 14 10:57:01.558780 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: spawned uWSGI master process (pid: 80473) Jul 14 10:57:01.558983 dv-cinder-822821-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 10:57:01.559751 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: spawned uWSGI worker 1 (pid: 80483, cores: 1) Jul 14 10:57:01.560100 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: spawned uWSGI worker 2 (pid: 80484, cores: 1) Jul 14 10:57:01.560257 dv-cinder-822821-5-iscsi devstack@keystone.service[80473]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 10:57:06.995769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:57:06.995769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: from cryptography.utils import int_from_bytes Jul 14 10:57:06.996609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:57:06.996609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: from cryptography.utils import int_from_bytes Jul 14 10:57:06.997514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:57:06.997514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: from cryptography.utils import int_from_bytes Jul 14 10:57:06.997980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:57:06.997980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: from cryptography.utils import int_from_bytes Jul 14 10:57:07.045862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:57:07.046047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:57:07.046239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] command line args: [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:57:07.046506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:57:07.046682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ================================================================================ {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:57:07.046939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] admin_token = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.047254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:57:07.047336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] config_dir = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.047468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:57:07.047561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.047663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] command line args: [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:57:07.047745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] config_source = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.047849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] debug = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.047931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:57:07.048047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ================================================================================ {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:57:07.048376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] admin_token = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] config_dir = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.048980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] config_source = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] debug = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_config_append = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.049910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_dir = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_file = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.050512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_config_append = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_dir = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_file = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.051987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.052949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.053900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] publish_errors = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.054805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] publish_errors = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] standard_threads = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.055996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] use_journal = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] use_json = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] standard_threads = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] use_stderr = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] use_syslog = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.056878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.057432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] use_journal = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.057605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] use_json = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.057829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] use_stderr = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.057946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] use_syslog = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.058020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:57:07.058252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.058990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.059998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.060973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.061952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.062926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.063883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.064985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.065946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.066015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.066087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.066161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.067934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.068970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.069958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.070981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.071949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.072886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.073981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.074944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.075950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.076916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.077870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.078969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.079981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.080943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.081719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.082553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.082723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.082786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.082984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.external = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.083892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.password = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.084926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.external = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] auth.token = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.085951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.password = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] auth.token = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.086958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.087887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.088906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.089912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.090982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.091923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.092940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.093939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.094964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.095936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.096966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.097089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.097215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.097337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.097467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.098515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.098656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.098782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.098914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099206 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.099828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.100953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.101930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.102918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.103941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.104931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.105920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.106908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.107915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.108894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.109885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110128 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.110992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.111920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.112935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.113074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.114787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.115887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.116952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.117897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.118887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.119922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] role.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] role.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.120922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.121922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] role.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] role.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.122920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.123929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.124909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.125894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.126870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.127986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.128733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.129986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.130973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.131906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.132923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.133957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.connection = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.134983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.135872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.connection = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.136952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.137952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.138793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:57:07.139851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80484) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:57:07.164477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.164703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.164927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.165494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.165739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.165969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.173250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.174354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.201969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.202198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.203147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.203384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.344755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.345046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.394640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.394848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.395004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.395172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.459861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.460900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.461019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.461153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.461280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.461280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.485926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.486956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.487631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.498152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.498558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.510779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.511992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.512122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.512246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.512369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.512369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.512610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80484) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:57:07.529489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.529701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:57:07.529985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.530119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.530241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:57:07.530369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:57:07.530507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.530700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.530825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:57:07.530953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.531226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.531356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.531508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.531630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.531755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.532239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.543129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.543420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.543665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.543791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.543945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.544072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.544212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.544481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.552196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.552782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.555466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.555744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.556301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.556957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.557986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.558119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.558246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.558412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.558682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.584839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.585552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.586421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.586697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.587190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.587473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.591899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.592176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.592459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.592722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.593001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.593281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.600491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.600725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.601371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.601655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.605147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.605432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.605709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.605986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.606261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.606936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.607071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.607253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.607527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.607794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.608059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.625477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.625752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.626651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.626932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.630757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.631821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.634696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.634987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.635446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.635745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.640087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.640897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.643669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.645070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.648128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.648425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.648715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.648862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.648996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.649845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.650121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.669363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.669648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.669918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.670046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.670170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.670304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.670465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.670730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.678194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.678696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.679994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.680265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.680519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.680799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.683455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.683734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.684131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.684821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.685087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.697953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.698228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.698378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.698504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.698633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.698759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.698886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.699968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.719670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.719846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.723530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.723697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.726794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.726937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.728403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.728536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.730149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.730291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.730429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.730556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.730683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.730809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.737979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.738208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.739651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.740117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.742818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.743926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.744176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.752823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.753103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.753377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.753506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.753708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.754090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.761145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.761424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.761701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.761975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80484) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:57:07.762269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.762920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.763901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.764819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.765030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.765309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.765579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80484) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:57:07.803556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56217c3849c0 pid: 80483 (default app) Jul 14 10:57:07.805066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56217c3849c0 pid: 80484 (default app) Jul 14 10:57:07.806923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951b90-083c-4ae2-9304-b7bbf628e8ef None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.807245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951b90-083c-4ae2-9304-b7bbf628e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.807535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951b90-083c-4ae2-9304-b7bbf628e8ef None None] PATH_INFO: `/v3` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.811772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Thu Jul 14 10:57:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:16.531087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:16.531935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:16.531935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:16.537054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:16.542543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:16.542990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:16.543083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:16.613392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: INFO dbcounter [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] Registered counter for database keystone Jul 14 10:57:16.634640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG oslo_db.sqlalchemy.engines [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80483) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:57:16.641003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writer thread running {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:57:16.821459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG passlib.handlers.bcrypt [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80483) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:57:16.821799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG passlib.handlers.bcrypt [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80483) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:57:16.876362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:16.877998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:17.001114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 10:57:17.201965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:17.272421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: INFO dbcounter [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] Registered counter for database keystone Jul 14 10:57:17.292204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG oslo_db.sqlalchemy.engines [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80484) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:57:17.298304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writer thread running {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:57:17.421249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047e1df-693e-48a0-9839-34e3c83a761a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:17.425551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:17.426210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:17.426495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:17.426771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:17.434923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:17.486515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING keystone.common.rbac_enforcer.enforcer [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:57:17.486860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b129b4-21fb-451e-973a-68eb8bf2c36e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:17.496461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Thu Jul 14 10:57:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 10:57:17.502489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:17.516997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0da54-c048-4538-bdd9-ca8895f59048 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:17.521935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:17.523060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:17.523382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:17.523665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] PATH_INFO: `/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:17.525169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] RBAC: Authorizing `identity:get_project(project_id=7da05e4cf2b444729a9cbe3b6d5db535)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:17.578521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:57:17.579082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23131b-0745-4b6d-b758-ef8c594bc97e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:17.584052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Thu Jul 14 10:57:17 2022] GET /identity/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:19.591410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eafd3e-7c5e-4a4c-b564-f4d09b369b75 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.591932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eafd3e-7c5e-4a4c-b564-f4d09b369b75 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.592034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eafd3e-7c5e-4a4c-b564-f4d09b369b75 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.596075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:19.601998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.602417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.602590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.621564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:19.623093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:19.631374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 10:57:19.653609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eafd3e-7c5e-4a4c-b564-f4d09b369b75 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.666453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eafd3e-7c5e-4a4c-b564-f4d09b369b75 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.670557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.671217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.671503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.671780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.679483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.681263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78df79f-d834-4eb9-ba32-bf4e6f5a4a15 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.689886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:57:19.694962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.703914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c016f-fbd6-4633-817b-89ab9ed8ebd3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.708807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.709661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.709947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.710225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.711646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.713462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873524a-c241-4949-bc48-94f0a7e0320d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.715023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:21.804816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec7612-cb71-41d2-bb51-98897cfd4ee2 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.805361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec7612-cb71-41d2-bb51-98897cfd4ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.805361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec7612-cb71-41d2-bb51-98897cfd4ee2 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.806739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:21.811937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.812329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.812505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.816752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308eca5-2d68-4649-8399-8917fa78f57d None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.817089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308eca5-2d68-4649-8399-8917fa78f57d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.817379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308eca5-2d68-4649-8399-8917fa78f57d None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.818676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:21.823872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.824162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.824444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.829604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:21.830813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:21.864683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG passlib.handlers.bcrypt [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80484) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:57:21.865098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG passlib.handlers.bcrypt [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80484) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:57:21.911383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:21.912380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:21.913007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:21.935286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:21.959696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.015469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:22.038764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.039778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6a87c-2cac-4806-bf7f-ee5130724956 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.073883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.074609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.074895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.075202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.087022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:22.098162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:22.099277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:22.100257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:22.105287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.141415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60d724-0a2e-4f79-bd60-f6a96fca6c39 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.143740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.178438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.179380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.179669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.179948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.189218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.208637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83193090-bf0c-42bf-aac8-00e837fb290e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.242050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.243055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.243374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.243651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.244397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:22.244704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.246104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.249350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.250860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.258285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:22.263813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.350082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 8/16] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:57:22.355437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.358175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.364388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba46d7d-f7c7-4fb2-965b-614c2c95d643 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.398201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.399069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.399380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.399656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.400861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.402338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.403984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 9/17] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:22.409586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.423936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.454764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c238-829d-4d3e-ab73-7eedfe8d4fbe None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.488548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.489338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.489622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.489898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.508650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.509023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c32703-6895-462b-9aa7-cdb1359a4ae0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.510118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:22.531097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:22.532350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:22.533344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:22.533344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:22.533344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:22.533344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:22.533344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 9/18] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:22.538563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.544703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.545503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.545786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.546063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.547270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.548718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.551000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 10/19] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:22.556491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.595832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.636221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.639869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68a77b-2666-4478-9403-0161f6e254f0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.655683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694278d-2d74-464f-bd45-62c9aa293192 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.672634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.673308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.673596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.673873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.674602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:22.675441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.676946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2df9d9-efd4-4feb-81e9-360e82bde58e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.691243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.692138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.692420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.692697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.693529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.domain.id=default, user.id=23fec8c4e37c41799ff49ae249e16e30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:22.693842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.694799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 10/20] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:22.695315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333888c1-fb9e-4361-9d64-2fc9dd851cd8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.702374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 11/21] 10.222.0.59 () {56 vars in 1322 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/role_assignments?user.id=23fec8c4e37c41799ff49ae249e16e30&scope.domain.id=default&role.id=beb4a709a14b415db991ad1980b406a9 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:25.048341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440373f4-1737-46a0-9dde-7fcfb41229d0 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.049415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440373f4-1737-46a0-9dde-7fcfb41229d0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.050081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440373f4-1737-46a0-9dde-7fcfb41229d0 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.051871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 11/22] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:25.057103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.057953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.058439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.087362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:25.088914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.095770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466e65b-8e5c-4023-af2c-c052e51afe17 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.096155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466e65b-8e5c-4023-af2c-c052e51afe17 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.096684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466e65b-8e5c-4023-af2c-c052e51afe17 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.098249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 12/23] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:25.106006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.106587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.107090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.136173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:25.137703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.233425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 12/24] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:25.276839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.285248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 13/25] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:25.354629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.410024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:25.411671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:25.459703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc5fba-dbd9-4897-8c76-40149c3a3878 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.466266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8b8e-ed90-49d6-ae1b-cc2b1f8902ad None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.512369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.513058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.513351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.513636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.514394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.515879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ae821-a3a2-439f-b887-8e53006dc9d6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.532958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.533774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.538795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.546377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.551875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.553435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-114b4b16-0387-428a-8d64-a965ae173766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:25.573124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:25.575728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 13/26] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ffe25-f79a-4933-8941-a87c5a858456 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ffe25-f79a-4933-8941-a87c5a858456 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.576531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ffe25-f79a-4933-8941-a87c5a858456 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.578672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 14/27] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:25.582999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ffe25-f79a-4933-8941-a87c5a858456 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.701275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ffe25-f79a-4933-8941-a87c5a858456 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.731602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:25.734449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 14/28] 10.222.0.59 () {60 vars in 1131 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/roles => generated 219 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:25.736052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896e7f-aa3c-4cd5-b1af-bc7dd59c6850 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.736413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896e7f-aa3c-4cd5-b1af-bc7dd59c6850 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.736693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896e7f-aa3c-4cd5-b1af-bc7dd59c6850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.753089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238597-a261-4310-84cf-4925132f573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.753862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238597-a261-4310-84cf-4925132f573a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.754156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238597-a261-4310-84cf-4925132f573a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.754435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238597-a261-4310-84cf-4925132f573a None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.755199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238597-a261-4310-84cf-4925132f573a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:25.755510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238597-a261-4310-84cf-4925132f573a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.756921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238597-a261-4310-84cf-4925132f573a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.758851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ce896e7f-aa3c-4cd5-b1af-bc7dd59c6850 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:25.760632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ce896e7f-aa3c-4cd5-b1af-bc7dd59c6850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.794794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 15/29] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:25.796278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a966a7-6b4a-4b92-bd5e-6280949a4b65 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.796546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a966a7-6b4a-4b92-bd5e-6280949a4b65 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.796822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a966a7-6b4a-4b92-bd5e-6280949a4b65 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.798198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 16/30] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:25.808162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a966a7-6b4a-4b92-bd5e-6280949a4b65 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.880266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 15/31] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:25.882742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.885460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.885951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.920452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:25.926776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.967691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:25.995084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a966a7-6b4a-4b92-bd5e-6280949a4b65 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.008904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 16/32] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:26.009643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.035996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.036997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.037496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.037979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.038877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.040583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.127033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04533b-0605-4cf3-a75a-9be08c44621f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.169723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.170577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.170861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.171236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.195098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:26.204688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:26.205525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:26.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 17/33] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:26.212615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.243440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.282223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 17/34] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/projects => generated 301 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:26.291450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.334905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263addc8-f2aa-4e19-8f97-5122e149c0bd None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.369471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.380040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.381301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.381749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.382345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.383309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.385586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a745c11-9b10-4758-97d4-37f9489b0956 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.458153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c56b-1e24-474f-8e05-1eae49a9ac3f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.497405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.498079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.498375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.498665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.499557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:26.499883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.501431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.510952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 18/35] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:26.515221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 18/36] 10.222.0.59 () {60 vars in 1131 bytes} [Thu Jul 14 10:57:26 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:26.516527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.614987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b81981-d0ef-4180-b555-6838e6a552c9 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.615415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b81981-d0ef-4180-b555-6838e6a552c9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.615776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b81981-d0ef-4180-b555-6838e6a552c9 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.617393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 19/37] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:26.624249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.624643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.624998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.652277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:26.653539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:26.737901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 20/38] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:26.761546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977639a3-ae65-40b8-8a1f-134bb854de47 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.762912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.763867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.784517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.800363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.801245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.801535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.801817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.803153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.804608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51a2e3-7e2c-44c5-b069-32fbc7815fc2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.806252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 19/39] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:26.807669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af77c1-61a2-48c7-a10a-f42d60de6161 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.807964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af77c1-61a2-48c7-a10a-f42d60de6161 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.808246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af77c1-61a2-48c7-a10a-f42d60de6161 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.809632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 20/40] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:26.811720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af77c1-61a2-48c7-a10a-f42d60de6161 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.869450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a438-8a0b-4494-b0b7-921ea6feb406 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.904180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.904850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.905140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.905422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.906430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.907843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.915393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af77c1-61a2-48c7-a10a-f42d60de6161 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.949430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.950224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.950516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.950802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.952223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.953686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38774b-f75b-4e9b-9677-738909758c46 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.955473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 21/41] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:26.957214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.957507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.957794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.964458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 21/42] 10.222.0.59 () {60 vars in 1131 bytes} [Thu Jul 14 10:57:26 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:26.965288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.976439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:26.978321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:27.015986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:27.059093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:27.061586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 22/43] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:27.086258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c5d61-53cb-41b7-8646-0526afdf9337 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:27.087235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:27.124794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:27.126409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.126409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.126409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] PATH_INFO: `/v3/domains/default/users/23fec8c4e37c41799ff49ae249e16e30/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.129042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=23fec8c4e37c41799ff49ae249e16e30, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:27.130525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:27.177057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 22/44] 10.222.0.59 () {58 vars in 1361 bytes} [Thu Jul 14 10:57:26 2022] PUT /identity/v3/domains/default/users/23fec8c4e37c41799ff49ae249e16e30/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:27.204201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b53f1c-ef01-4ba6-ab49-65a166d4a80d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:27.242237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:27.243060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.243376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.243656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.244903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:27.246396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8225d-6ead-448b-aea3-8b76bf1956a4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:27.248093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 23/45] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:27.253972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:27.364119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3812d3-0696-46c1-a4ac-3ecf4d146369 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:27.399004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:27.399709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.400002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.400284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.401000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:27.402469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e731c97-c5cf-48ea-a11c-45aeeb39ea8c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:27.407200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:27.502435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 23/46] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:57:27 2022] POST /identity/v3/projects => generated 312 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:29.216193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d748-a30f-4f4b-8852-13a6f960dcc4 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.216678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d748-a30f-4f4b-8852-13a6f960dcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.216761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d748-a30f-4f4b-8852-13a6f960dcc4 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.218298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 24/47] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:29.224010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92c10-5947-4bbb-8a25-e65f617a211d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.224435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92c10-5947-4bbb-8a25-e65f617a211d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.224698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92c10-5947-4bbb-8a25-e65f617a211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.242189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-41c92c10-5947-4bbb-8a25-e65f617a211d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:29.243424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-41c92c10-5947-4bbb-8a25-e65f617a211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:29.415206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 24/48] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:29.437867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d748-a30f-4f4b-8852-13a6f960dcc4 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.474325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.537260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c915707-e63e-4b12-b380-20aa7b6541b2 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.537552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c915707-e63e-4b12-b380-20aa7b6541b2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.537842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c915707-e63e-4b12-b380-20aa7b6541b2 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.539247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 25/49] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:29.541638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d748-a30f-4f4b-8852-13a6f960dcc4 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.543976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.544336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.544616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.561133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:29.562326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:29.575887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.576691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.576981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.577263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.588402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:29.596009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:29.596952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:29.597719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 25/50] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:29.604376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.630013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 26/51] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:29.653333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.697105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.701797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.703545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb347f-ef22-44fd-8ce2-c4b062ee9e16 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.737582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.738262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.738555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.738841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.739632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:29.739948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.741350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.749343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 26/52] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:29.752244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec86beb-4249-4e89-80d4-1f536e8b7cf0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.755430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.787113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.788622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.788622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.788622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.789798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.791386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.793036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 27/53] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:29.798654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.855807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14a827-a245-4b8d-abf6-f1ba19b2a5c3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.888845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.889719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.890004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.890283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.891506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.893060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.895577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 27/54] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:29 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:29.900924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.902083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc931b-58eb-463c-a18b-f9a375d0fff5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.919048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.925867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.938369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.939037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.939420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.939627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.941262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.942825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbe50-03e7-4af2-88f1-0b3ded02b95d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.000835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd452b5-5094-4535-8113-38fc69fd7ef6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.034485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.035309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.035602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.035881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.037065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.040449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.040449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 28/55] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:30.045990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.069113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 28/56] 10.222.0.59 () {60 vars in 1140 bytes} [Thu Jul 14 10:57:29 2022] POST /identity/v3/projects => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:30.145281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:30.154684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a9a2-1f9f-4d06-a837-7bf5960f1ab6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.194828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.195571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.195867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.196153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.197045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.domain.id=default, user.id=23fec8c4e37c41799ff49ae249e16e30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:30.197368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.198868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44a19b-e5eb-44f7-aa23-519c59208220 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.205445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 29/57] 10.222.0.59 () {56 vars in 1322 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/role_assignments?user.id=23fec8c4e37c41799ff49ae249e16e30&scope.domain.id=default&role.id=beb4a709a14b415db991ad1980b406a9 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:32.097699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779305-ec75-4c8d-b575-ffdc55e5d0c8 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.098207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779305-ec75-4c8d-b575-ffdc55e5d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.098310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779305-ec75-4c8d-b575-ffdc55e5d0c8 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.099950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 29/58] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:32.105134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.105572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.105701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.122574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:32.123805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.191584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 30/59] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:32.216289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b779305-ec75-4c8d-b575-ffdc55e5d0c8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:32.313979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b779305-ec75-4c8d-b575-ffdc55e5d0c8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.347460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.348289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.348525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.348800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.349971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.351490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccebdc-6844-4039-ba7a-30088cc0b691 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.353073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 30/60] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:32.358482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:32.446814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:32.461259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67651bc9-27eb-4cfd-8a5d-f1a1e0ceed5f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.495223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.495897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.496186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.496468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.497187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.498699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8582ae-505a-4ee8-8180-1cf1f73e92b2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.608668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 31/61] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:32 2022] POST /identity/v3/users => generated 275 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:35.879427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3701-579a-4ba3-a0ac-b33fbc051a17 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:35.879934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3701-579a-4ba3-a0ac-b33fbc051a17 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:35.880202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3701-579a-4ba3-a0ac-b33fbc051a17 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:35.881438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 31/62] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:35.886807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21695-a5eb-4cc6-9144-a3988a10970a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:35.887252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21695-a5eb-4cc6-9144-a3988a10970a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:35.887466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc21695-a5eb-4cc6-9144-a3988a10970a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:35.912258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3dc21695-a5eb-4cc6-9144-a3988a10970a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:35.913507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3dc21695-a5eb-4cc6-9144-a3988a10970a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.006157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016d7a6-02df-4ed5-9a3a-776bdc8899d6 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.006485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016d7a6-02df-4ed5-9a3a-776bdc8899d6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.006724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016d7a6-02df-4ed5-9a3a-776bdc8899d6 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.008220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 32/63] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:36.009424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 32/64] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:36.015292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.015583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.015910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.019177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b150b-83a0-4dcb-9ad7-a699f210c52e None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.019476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b150b-83a0-4dcb-9ad7-a699f210c52e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.019762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b150b-83a0-4dcb-9ad7-a699f210c52e None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.021093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 33/65] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:36.029407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.029695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.029980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.048143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.050311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.053776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.059150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.087990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.134403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 34/66] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:36.139869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 33/67] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:36.142296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.147507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.164685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.277307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9341a0-d227-4a82-a38f-1af5635cb24f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.292013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d34c6-5ee0-4821-81e8-caf0d1a14bba None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.315923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.316731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.317017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.317293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.327781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:36.329930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.331653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.335552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.335552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:36.336004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:36.336954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 35/68] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:36.342757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.350079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-163bc0ed-62d5-4097-9641-2b4b148c5b98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:36.358539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:36.359671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 34/69] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:36.361279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c787-15f6-4048-bf84-cd8b75cf9c4a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.362912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c787-15f6-4048-bf84-cd8b75cf9c4a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.362912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c787-15f6-4048-bf84-cd8b75cf9c4a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.363256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 35/70] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:36.364337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02b28-67c9-4d1f-81b6-598c16595632 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.364895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02b28-67c9-4d1f-81b6-598c16595632 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.365012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02b28-67c9-4d1f-81b6-598c16595632 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.366154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 36/71] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:36.367285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f02b28-67c9-4d1f-81b6-598c16595632 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.398224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.405971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.445153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08553067-a1b7-4b81-bdbe-54e658779011 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.472841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f02b28-67c9-4d1f-81b6-598c16595632 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.479987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.480796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.481080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.481358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.491542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:36.499069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:36.499887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 36/72] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:36.506340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.507028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.507345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.507587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.507861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.508614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:36.508926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.510304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4113a94-67d5-4658-a4c3-a01b949c4be5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.523003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 37/73] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:36.523003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc306-85cd-4a6b-96dc-7c0bd9e4e469 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.523003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc306-85cd-4a6b-96dc-7c0bd9e4e469 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.523003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cc306-85cd-4a6b-96dc-7c0bd9e4e469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.543212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f27cc306-85cd-4a6b-96dc-7c0bd9e4e469 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.544790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f27cc306-85cd-4a6b-96dc-7c0bd9e4e469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.604160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865bff-9917-4392-8e82-943014aaed4d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.613161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 38/74] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:36.614396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.614699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.614981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.628827 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.632150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.633370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.641014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.641699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.642028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.642316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.643689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:36.644070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.645497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.647727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.655450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 37/75] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:36.656403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.702705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 39/76] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:36.703920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.766249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b1c8-7577-4ac0-a66a-a98cdd03ff10 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.801345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.802026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.802320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.802606 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.803407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:36.803723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.805131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.806168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b0a97-9a94-434e-81b0-1068777924e8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.813352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 38/77] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:36.814244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.845381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.847099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.847418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.847696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.848914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.850373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.852049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 40/78] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:36.852049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.860440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.875374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.914083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dcc9d-e8d5-4f2d-a2ed-81d6e08da115 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.947594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.948392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.948679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.948960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.950145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.951666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.953313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 39/79] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:36.954074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.959325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fae36-29c3-455d-a0b2-c14926a0924d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.994176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.995051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.995382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.995664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.010913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.012444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.014282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 41/80] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:37.014896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.054043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03047b7a-2160-4329-b1db-793ff3b6311d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.068566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.088843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.089645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.089936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.090218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.093537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.100694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:37.108287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:37.109161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:37.110170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 40/81] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:37.111054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.124912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23183-2c00-4c85-adb2-4e805bd32586 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.161044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.161924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.162215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.162493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.163998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.165528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.167143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 42/82] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:37.167143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.213454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322e6da-70a5-4e00-ad36-b0210223bb3e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.247344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.248006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.248299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.248582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.272888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94922ec2-5426-4a73-b73c-bd3057f692ba None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.291203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.305484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.306156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.306446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.306723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.307465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.308922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.316104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.317588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.386875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.392483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 43/83] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:37 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:37.393890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.429148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 41/84] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:37.432203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.513060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d366a04-3055-41a1-a872-31ed84f4c7ab None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.519158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.549639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3f6a-0e31-49b3-be5b-3396533f4d03 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.555339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.556405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.557004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.557004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.586500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.587332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.587621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.587900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.588767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:37.588952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.590344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.599095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 42/85] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:37.599960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.638955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.641309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.644498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.720818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021015c6-9bb3-4f1f-8827-91248b22ef75 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.767356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 44/86] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:37.768471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.768704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.769224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.769524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.770085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.775026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.870529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.872365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.894773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.900504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86973c5-e8d0-44f1-b3ad-eb7739318599 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.944698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.946266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.946773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.947317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.949005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.950907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.953276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 45/87] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:37.955627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.026306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 43/88] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:38.028280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.078431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.097460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc081e1-ed3c-4abf-88c7-25fefb739b5e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.145742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.146901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.147240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.147528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.149057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.151300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.153706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 46/89] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:38.160266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.167481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f8ef5-3306-41f6-9b3f-e585f30eebfd None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.197675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.229305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.230586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.231193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.231664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.233136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.235020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.239104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 44/90] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:38.240344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.290653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea9288-7805-413f-95d9-329f44a8893a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.334064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.334736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.339259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.339259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.339259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=23fec8c4e37c41799ff49ae249e16e30) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:38.339259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.339259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.351216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 47/91] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/role_assignments?user.id=23fec8c4e37c41799ff49ae249e16e30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:38.352471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.398300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7130c-c650-4cda-a934-12ee695fd4fa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.402522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.480747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.483565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.484285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.485499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.489594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.494738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.499724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 45/92] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:38.501359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.580093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.584806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51dc95-7087-447b-a1ce-80d4f5f9ce1a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.624605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.625295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.625590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.625913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.670505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4fb03-e18e-4ae6-a48b-5c177d682e0d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.702871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.704489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.747821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.748938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.749426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.749995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.751059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:38.751603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.753546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.768659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 46/93] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=d5708cbdc7894157a71389bd8517df5f => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:38.769983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.838846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.839517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.851653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 48/94] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:38.858856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.975598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369832-e26a-4842-8595-c153109bdde6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.043015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.043826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.044026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.044317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.045173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:39.045490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.046965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.053295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c0773-2e3b-418b-b4fb-71ed0c5824fd None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.058655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 47/95] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:39.093467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.094267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.094607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.094894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] PATH_INFO: `/v3/projects/2fddb753d45d403fb207a6b873598723` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.096757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] RBAC: Authorizing `identity:get_project(project_id=2fddb753d45d403fb207a6b873598723)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.098299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81ef0a-f05b-4575-a938-21a1dadbf88a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.100370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 49/96] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/projects/2fddb753d45d403fb207a6b873598723 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:39.106755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.236498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.239636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60603c0-3144-4bb6-8892-ce842e870c4d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.284527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.285341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.285630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.285910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.286752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=2fddb753d45d403fb207a6b873598723, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:39.287110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.288627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba4b30-1207-42cb-a0f0-a1048b6ab2cb None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.294760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 48/97] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:39 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=2fddb753d45d403fb207a6b873598723&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:40.091314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ae9a-02ff-4441-8c78-bf2c65e76fd2 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.091695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ae9a-02ff-4441-8c78-bf2c65e76fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.091887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ae9a-02ff-4441-8c78-bf2c65e76fd2 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.093289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 50/98] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:40.099265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.099710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.099881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.108728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838eaf3-bced-4f8e-83d2-b968caa0f988 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.109239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838eaf3-bced-4f8e-83d2-b968caa0f988 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.109417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838eaf3-bced-4f8e-83d2-b968caa0f988 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.110738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 51/99] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:40.124328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.124623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.124912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.127764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.128954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.163367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.164581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.247305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 49/100] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:40.249283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.262891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 52/101] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:40.296242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.332436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.405095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.419566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc321539-42b0-44c7-bc34-1f0729a9e6ff None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.483484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.484335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.484630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.484910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.491957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.493454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.495278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 50/102] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:40.503798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.651149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f95971-d5c9-434f-9272-943dcbdcfa03 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.669228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cf841-bde1-416a-8664-4b336a6c2f4d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.709661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.710345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.710640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.710928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.711692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.716690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.773703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.774527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.774819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.775100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.797507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:40.798534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 51/103] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/projects => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:40.806469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:40.807444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:40.808199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:40.811945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 53/104] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:40.818924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.849174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.999994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4db6f-9624-4ecf-953f-8315410f1af7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.039810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.040496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.040844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.041069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.041832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:41.042148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.043586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e321-a2a2-4e27-aa63-310fd5274ac5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.054159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 52/105] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:41.059940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.134556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.205565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766bd4d-9a06-468b-9138-b2d9e3898001 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.252557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.253376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.253668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.253948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] PATH_INFO: `/v3/users/0724864cb10741be9a9549f091020c1c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.267822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7de7c-0fee-49fd-81be-5aa829c8346a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.268303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7de7c-0fee-49fd-81be-5aa829c8346a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.268493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7de7c-0fee-49fd-81be-5aa829c8346a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.269797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 53/106] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:41.273536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] RBAC: Authorizing `identity:get_user(user_id=0724864cb10741be9a9549f091020c1c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.274892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.275068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.275261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.275511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.276796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 54/107] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/users/0724864cb10741be9a9549f091020c1c => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:41.286055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.298273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:41.299649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.474833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 54/108] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:41.495888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b28f8-3ea6-45ef-ac04-1c2e59537b5b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.515545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.528202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.538131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.580669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.581394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.581682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.581972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.696623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33500a-8e3c-4b3d-bd07-07bdd6583a6f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.765363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.766749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.766987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.767330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.767442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.769218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.790990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:41.807825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:41.808698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:41.809478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 55/109] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:41.816315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.940636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.002451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0a938-8f88-40e9-a430-01252f3e37c4 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.011953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 55/110] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:42.018338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.049405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.050458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.050964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.051459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.051558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.052508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:42.053014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.054617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.063753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 56/111] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:42.069732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.199886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661d688-44f7-4618-bb3f-49f01317345e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.203899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64c00-c6a6-4daf-b690-9d2f860b0f20 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.229819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.237933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.239151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.239617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.244244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.244244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.244244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219cbba-ac17-4c3a-bd0c-c4b73dbf8361 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.244877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 57/112] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:42.246259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3ac-634f-4dee-a048-5f16ce2205e4 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.246726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3ac-634f-4dee-a048-5f16ce2205e4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.251348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3ac-634f-4dee-a048-5f16ce2205e4 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.253067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 58/113] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:42.254363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd3ac-634f-4dee-a048-5f16ce2205e4 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.303937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.304858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.305240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.305597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.311905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.313494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cb46d-414d-4150-95a8-febb9abbe5f7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.316098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 56/114] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:42.317845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a49e2-2496-4306-b0b2-774fbc3927f5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.318302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a49e2-2496-4306-b0b2-774fbc3927f5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.318727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a49e2-2496-4306-b0b2-774fbc3927f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.355988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-984a49e2-2496-4306-b0b2-774fbc3927f5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:42.357498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-984a49e2-2496-4306-b0b2-774fbc3927f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.370063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd3ac-634f-4dee-a048-5f16ce2205e4 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.415295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.416414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.416813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.417431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.419652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.421230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.423087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 59/115] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:42.424632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.456521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 57/116] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:42.458101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8255-1c42-4f0e-8d45-02f81c9300aa None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.458497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8255-1c42-4f0e-8d45-02f81c9300aa None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.458840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8255-1c42-4f0e-8d45-02f81c9300aa None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.460228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 58/117] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:42.465009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb8255-1c42-4f0e-8d45-02f81c9300aa None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.481558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.515588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.557423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de8093-cabf-4fa9-8311-07196e0d4804 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.576025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb8255-1c42-4f0e-8d45-02f81c9300aa None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.593202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.593966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.594262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.594548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.595518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=0724864cb10741be9a9549f091020c1c) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:42.595957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.597472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7ccdc-d449-4d26-9ca0-e2798ee4c1f6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.609573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 60/118] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/role_assignments?user.id=0724864cb10741be9a9549f091020c1c&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:42.609573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.609573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.609573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.622980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.624160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:42.624269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.624445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.624723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/23fec8c4e37c41799ff49ae249e16e30/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.625392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.627733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, user_id=23fec8c4e37c41799ff49ae249e16e30, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.629610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca17b6-fdb3-45a9-bb31-b295103f5c54 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.707421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 61/119] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:42.708587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.727521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 59/120] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:42 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/23fec8c4e37c41799ff49ae249e16e30/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:42.739402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940469-4e20-41b0-b182-7496dbfc5661 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.739950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940469-4e20-41b0-b182-7496dbfc5661 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.740461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a940469-4e20-41b0-b182-7496dbfc5661 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.742592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 60/121] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:42.748568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.750603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.751080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.779361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:42.781137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.827223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.878579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ead721-f860-4f26-a403-ba35255b5b53 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.878969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 61/122] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:42.879871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.889775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.945072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.946259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.946770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.947285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.959937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:42.969980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:42.970910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:42.971686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:42.974741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 62/123] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:42.975808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.004585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86145bea-93c4-4df9-bd4a-455b5c33c106 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.047080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.048049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.048348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.049400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.063490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:43.077346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:43.078222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:43.079009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 62/124] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:43.079841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.117744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2248c1-2149-4834-a256-ad74c8cde339 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.140980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.155874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.162123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.163089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.163401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.163680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.177118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:43.191525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:43.192469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 63/125] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:57:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:43.193232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.274123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46388d-46c0-4a45-aa34-c87d28469583 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.309741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab963-d479-4e02-bd85-5d72b6b8e3c9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.346251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.346869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.347184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.347475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.348247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:43.348570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.349994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.353393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.354093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.354383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.354665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.355452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:43.356251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.358042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.359224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 63/126] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.360262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.372198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 64/127] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.373135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.420791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.485118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369cd65-9385-491a-86c5-2cffff9735d6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.529967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.531207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.531504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.531784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.533193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.535041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.537238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 65/128] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.538022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.577301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c480a1-cf69-4de7-aa7f-766c9f1c988a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.608510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.622885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.623919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.624476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.625478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.626442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:43.627665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.630348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4fe58-ec07-48eb-855d-df5477869dd7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.641456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 64/129] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.643574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c54f1-a1ef-48cc-8a97-e8d2af28ab9c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.644048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c54f1-a1ef-48cc-8a97-e8d2af28ab9c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.644961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c54f1-a1ef-48cc-8a97-e8d2af28ab9c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.651486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 65/130] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:43.652187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c54f1-a1ef-48cc-8a97-e8d2af28ab9c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.659328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5cda7-6205-41b6-b151-b46640e515db None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.669295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.697622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.698501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.698785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.699060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.701181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.702710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.705035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 66/131] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.705619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.791156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c54f1-a1ef-48cc-8a97-e8d2af28ab9c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.812555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec004f-7b86-412d-8049-fecdb93737d3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.836118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.837478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.838279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.839023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] PATH_INFO: `/v3/projects/2fddb753d45d403fb207a6b873598723` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.840879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=2fddb753d45d403fb207a6b873598723)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.842989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7a272-2b33-4daa-87aa-1f790f3ed8c2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.846210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 66/132] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/projects/2fddb753d45d403fb207a6b873598723 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.851035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.851035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.851035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.859375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.860314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.860703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.861779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.863061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.865582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.867716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 67/133] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:43.868719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.876624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:43.878467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.926589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.939669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.954044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 67/134] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:43.955365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.013163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e9b0-e269-4084-8591-a8ede933ea72 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.050375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.051436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.052127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.052127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.053458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.055000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.056989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 68/135] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:44.057889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.082123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78508c68-79d9-47fc-95d9-be7f0d3e9551 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.121973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.123087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.123551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.124025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] PATH_INFO: `/v3/projects/2fddb753d45d403fb207a6b873598723/users/a5a53c91bfd74877b8a19925a7e8da30/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.126379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fddb753d45d403fb207a6b873598723, user_id=a5a53c91bfd74877b8a19925a7e8da30, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.128921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.163017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4cc5a-1539-4762-83a0-b9c2911b507e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.178874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.209503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 68/136] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:43 2022] PUT /identity/v3/projects/2fddb753d45d403fb207a6b873598723/users/a5a53c91bfd74877b8a19925a7e8da30/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.210542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.217050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.218341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.218766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.219239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.221026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.222872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.225307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 69/137] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:44.226387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.301304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.425073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7bb2-f369-4562-92f2-19932cd60724 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.434814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c07f-9969-4ae2-b8e6-8620c4d85829 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.465598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.466749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.467100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.467414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.468622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.470918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.472809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 69/138] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:44.473695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.489723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.490896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.491376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.491821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/a5a53c91bfd74877b8a19925a7e8da30/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.494000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, user_id=a5a53c91bfd74877b8a19925a7e8da30, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.495669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.570137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 70/139] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:44 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/a5a53c91bfd74877b8a19925a7e8da30/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.571252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.641640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324250-4f60-4fdc-bb30-673ab00c485c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.671521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.699077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.699936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.700222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.700498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/a5a53c91bfd74877b8a19925a7e8da30/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.705961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, user_id=a5a53c91bfd74877b8a19925a7e8da30, role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.707491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaa9d8-7ff8-46a9-b086-56ead1edd90d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.724251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.793961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 70/140] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:44 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/a5a53c91bfd74877b8a19925a7e8da30/roles/d5708cbdc7894157a71389bd8517df5f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.809752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832e393-dade-4811-b36d-d988714953c7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.858609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.859354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.859648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.859930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.860901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.863293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db87445-865c-4a31-9374-d4d06dd257fc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.980998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 71/141] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/users => generated 283 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:45.094021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f9272-fd08-40b1-ae66-4e65fcadf874 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.094381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f9272-fd08-40b1-ae66-4e65fcadf874 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.094593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f9272-fd08-40b1-ae66-4e65fcadf874 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.096949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 71/142] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:45.102912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.103316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.103601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.121855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:45.123055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.204055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 72/143] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:45.227992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f9272-fd08-40b1-ae66-4e65fcadf874 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.360232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f9272-fd08-40b1-ae66-4e65fcadf874 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.409563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.410377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.410662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.410942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.427482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5ab0b3f9-19b9-4c7e-8dbb-ba3ddd776cc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:45.435435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:45.436291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:45.437070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 72/144] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:45.443499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.480558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.566022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4733f1e-9889-43d8-b3c9-047d0edfb62c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.609879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9fe21-f662-4975-8c0f-33d8104573ac None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.610231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9fe21-f662-4975-8c0f-33d8104573ac None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.610495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9fe21-f662-4975-8c0f-33d8104573ac None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.613117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 73/145] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:45.618868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.619173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.619449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.628487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.629183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.629463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.629738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.630510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:45.630813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.635382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.646993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 73/146] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:45.650657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.653936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:45.655472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.719428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.805874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 74/147] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 10:57:45.824015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a897c4-c70a-42c0-9c58-c7abd217f374 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.831498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.842696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.900581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.901407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.901688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.901958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] PATH_INFO: `/v3/users/0724864cb10741be9a9549f091020c1c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.908504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] RBAC: Authorizing `identity:get_user(user_id=0724864cb10741be9a9549f091020c1c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.910155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.911880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 74/148] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/users/0724864cb10741be9a9549f091020c1c => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:45.917192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.959889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92180b-9380-4a6e-a93a-d73bc6470514 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.001171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.002114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.002444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.002730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.014339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:46.024172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:46.025866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:46.027541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 75/149] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:46.032068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.062457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:46.146700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f077-797b-457d-bd47-4b32ef26eb80 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.196647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215b19-8253-470f-8005-decf6735b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.198950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215b19-8253-470f-8005-decf6735b3db None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.198950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215b19-8253-470f-8005-decf6735b3db None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.198950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215b19-8253-470f-8005-decf6735b3db None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.198950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215b19-8253-470f-8005-decf6735b3db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:46.198950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215b19-8253-470f-8005-decf6735b3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.200236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215b19-8253-470f-8005-decf6735b3db None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.210775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 76/150] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:46.216441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215b19-8253-470f-8005-decf6735b3db None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.219316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ba34-461c-4de4-98db-5e8d0b51e835 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.295643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.296533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.296922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.297206 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.301177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.302681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.304820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 75/151] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:46.310751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.332689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:46.351670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215b19-8253-470f-8005-decf6735b3db None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.362355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:46.389583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] PATH_INFO: `/v3/users/23fec8c4e37c41799ff49ae249e16e30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=23fec8c4e37c41799ff49ae249e16e30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.396398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 77/152] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:46 2022] GET /identity/v3/users/23fec8c4e37c41799ff49ae249e16e30 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:46.401714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.489527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184168b-293f-489d-b93b-8cca0d959c9e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.509321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b844-9135-4697-840e-af48220ca4a1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.541435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.542597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.543019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.543483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/0724864cb10741be9a9549f091020c1c/roles/a5df04a57f184f42ba005fa9eb51795c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.549206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.549675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, user_id=0724864cb10741be9a9549f091020c1c, role_id=a5df04a57f184f42ba005fa9eb51795c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.549875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.550176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.550458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.551802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.580103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:46.629562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.631055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.643289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 76/153] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:46 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/users/0724864cb10741be9a9549f091020c1c/roles/a5df04a57f184f42ba005fa9eb51795c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:46.743449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 78/154] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:46.748306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.885476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:46.902595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc944c8-f7c0-4146-b532-c43c0f78eeaa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.946434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.947355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.947643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.947921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.949203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.950755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44502a-10fa-4bba-a12a-72bb6add1d0c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.952546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 77/155] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:46 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:46.961444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.044003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:47.099384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e717c9-a7cb-4067-94e1-ec3643ff2aad None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.146584 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.147284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.147574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.147856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.148700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=23fec8c4e37c41799ff49ae249e16e30) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:47.149019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.150592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e996a-1714-4f47-b4b8-a091f46027fa None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.163276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 79/156] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:46 2022] GET /identity/v3/role_assignments?user.id=23fec8c4e37c41799ff49ae249e16e30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:47.465494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe28b8-ec8d-4eed-aeee-495ef0a54b9a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.465995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe28b8-ec8d-4eed-aeee-495ef0a54b9a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.466413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe28b8-ec8d-4eed-aeee-495ef0a54b9a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.467958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 78/157] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:47.474301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.475347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.475876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.496087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:47.497623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:47.580732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 80/158] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:47.618536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe28b8-ec8d-4eed-aeee-495ef0a54b9a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.775818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe28b8-ec8d-4eed-aeee-495ef0a54b9a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.819975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.821034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.821454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.821884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.837232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:47.845461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:47.846481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:47.847424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:47.849562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 79/159] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:47.855726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.951507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.053449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc82638-015b-4883-bdca-11a2f85e4f1e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.125251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.126987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.127492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.127974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.129256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:48.129802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.132187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5169aa-a826-427a-9622-8569720c8906 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.149650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 81/160] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:48.158458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.207506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.240924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf93d2-60ba-417f-88d6-a83b6a9da872 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.241388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf93d2-60ba-417f-88d6-a83b6a9da872 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.242333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf93d2-60ba-417f-88d6-a83b6a9da872 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.245094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 82/161] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:48.262733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.263220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.263696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.306080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:48.311150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:48.463153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 83/162] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:48.539473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.541990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85398-eff1-47ea-8da7-32e7c592a40d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.619433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.673128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.676263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.677061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.677810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.684250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.687106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.691351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 80/163] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 534 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:48.701186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.734820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd6dd-d933-4225-8e2c-d7f73ea7d9ad None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.805992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.807080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.807590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.808112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.832580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f50c0a69-1af1-4eca-94f5-cb0f3313973f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:48.840910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:48.841867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:48.842828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:48.852550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 84/164] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:48.854017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b590-522a-4842-b8c8-edf088029fbe None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.854452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b590-522a-4842-b8c8-edf088029fbe None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.855179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b590-522a-4842-b8c8-edf088029fbe None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.856637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 85/165] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:48.862715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.863217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.863749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.903197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:48.905358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:48.913038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.934088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6494-2c82-4649-a573-bec4f45b3e08 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.976438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.977819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.979244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.980239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.059400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.061164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.103146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 86/166] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:49.103146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.176861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:49.193785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:49.216001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 81/167] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:57:49.217158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.326982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee39c-243e-451c-a175-25d1c77ae6c0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.373587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.374764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.375286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.375796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.379002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e4af8-7d7a-4bd4-a934-3fd5f9f4f329 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.394288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:49.411265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:49.412202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:49.413083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:49.415511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 82/168] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:49.416529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.455230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.455230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.455230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.455230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.455694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:49.455694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.455826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.474501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 87/169] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:49.475361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.516437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:49.677354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8d280-6ebf-4757-a1d3-b97e2d139dca None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.684303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:49.698669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373980a-20e1-4665-af27-35b0d730f466 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.755518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.757244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.757750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.758358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] PATH_INFO: `/v3/projects/2fddb753d45d403fb207a6b873598723` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.760562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] RBAC: Authorizing `identity:get_project(project_id=2fddb753d45d403fb207a6b873598723)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.762314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.763233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.763538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.763912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.764374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.765320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:49.765826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.766978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 83/170] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/projects/2fddb753d45d403fb207a6b873598723 => generated 312 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:49.768179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.768285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.786360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 88/171] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:49.787549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.996756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780298c-98a6-4f67-ad1f-d4c5466de896 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.031449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.099534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.101039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.101507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.101961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.104115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.106865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.109764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 84/172] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:50.119804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.143782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1c9b-46da-4467-811e-2a986d775ee2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.237101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.237785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.238120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.238404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.239293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=2fddb753d45d403fb207a6b873598723, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:50.239611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.241206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.257583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 89/173] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:49 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=2fddb753d45d403fb207a6b873598723&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 608 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:50.258438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.294777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.377739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be43bc-015e-4af1-a44b-ea2f04667caa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.475934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.476908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.477197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.477478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] PATH_INFO: `/v3/users/a5a53c91bfd74877b8a19925a7e8da30` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.480067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] RBAC: Authorizing `identity:get_user(user_id=a5a53c91bfd74877b8a19925a7e8da30)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.481576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28569178-6afa-4e46-8f35-6ac8bb46308d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.483336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 85/174] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/users/a5a53c91bfd74877b8a19925a7e8da30 => generated 275 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:50.484591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e8627-45c4-4ec3-ba7a-98397dbbb058 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.484883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e8627-45c4-4ec3-ba7a-98397dbbb058 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.485163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e8627-45c4-4ec3-ba7a-98397dbbb058 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.490686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d599b-6de3-4133-97b7-2d9356fca617 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.491375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 86/175] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:50.492611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146a12-cc50-485a-9cb8-48ad5daa4ae0 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.492898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146a12-cc50-485a-9cb8-48ad5daa4ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.493179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146a12-cc50-485a-9cb8-48ad5daa4ae0 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.494608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 87/176] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:50.495328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1146a12-cc50-485a-9cb8-48ad5daa4ae0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.535209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.535872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.536157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.536431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.554562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.608229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.609707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f87e7-ff2a-4e49-ab57-884a50f564d1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.619311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1146a12-cc50-485a-9cb8-48ad5daa4ae0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.632572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.664792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.665467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.665749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.666028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.736557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 90/177] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:50.736557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0c3b-ec36-4b66-a1c1-09aba740ebe2 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.736557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0c3b-ec36-4b66-a1c1-09aba740ebe2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.736557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0c3b-ec36-4b66-a1c1-09aba740ebe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.742154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.743686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe386b-0c3d-4a86-8293-4d830e5c4392 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.750523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e6fe0c3b-ec36-4b66-a1c1-09aba740ebe2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:50.752159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e6fe0c3b-ec36-4b66-a1c1-09aba740ebe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.788251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.819472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 91/178] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:50.820665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.820947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.821221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.845587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 88/179] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:50.846221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:50.847324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717c66-838d-455d-a862-6a53f9be9248 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.847495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.847668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717c66-838d-455d-a862-6a53f9be9248 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.847902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0717c66-838d-455d-a862-6a53f9be9248 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.849319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 89/180] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:50.850371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79c08a-2b54-46cc-9b77-eb35a007cdee None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.850662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79c08a-2b54-46cc-9b77-eb35a007cdee None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.850939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79c08a-2b54-46cc-9b77-eb35a007cdee None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.852298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 90/181] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:50.853101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79c08a-2b54-46cc-9b77-eb35a007cdee None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.892403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.920285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 92/182] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:50.921216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.958550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79c08a-2b54-46cc-9b77-eb35a007cdee None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.994110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.994970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.995513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.995799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.996989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.998462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.000197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 91/183] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.001018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.018730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.037148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fdf4-f207-42bc-8e99-77f38bbf3eb0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.071100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.071852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.072134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.072406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.083046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-84613ed2-64d2-4f9a-815d-a9cbdba38da3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:51.090685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:51.091741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:51.092639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 93/184] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:51.093318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.093388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.093887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.105829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01342d-05ff-4999-ba53-06f5ca115600 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.110975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:51.112276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:51.139348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.140035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.140204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.140489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.140761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.142632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.144242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1b352-657f-4d96-a06d-dc121ad9286d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.145932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 92/185] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.147064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.147376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.147652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.164579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:51.165765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:51.182233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 94/186] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:51.182996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.239365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 93/187] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:51.251422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.262683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.289160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4673-d901-410a-84ef-ecbb8dd285af None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.323450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.324216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.324507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.324790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.335581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:51.340549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee806ed-9dc0-4a29-86c6-63f8527eccdc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:51.343891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:51.344770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:51.345527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 95/188] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:51.346188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.374850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.375295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.375531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.375812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.376689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.377519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:51.377831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.379312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.384988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 94/189] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.385755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.458103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761b49-5388-4ba5-9f5e-988c3ff3d4fa None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.482140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.498536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.499411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.499843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.500177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.501744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:51.502375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.503362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.512378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e02c2-5709-47e8-ad58-8233e8abcfc6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.518427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 96/190] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.518427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.563239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.564519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.564807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.565080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.565923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=a5a53c91bfd74877b8a19925a7e8da30) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:51.566234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.567712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.574858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 95/191] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/role_assignments?user.id=a5a53c91bfd74877b8a19925a7e8da30&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=d5708cbdc7894157a71389bd8517df5f => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.575502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.648833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.662535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e92b5-de30-4b7f-961d-4e0edc749985 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.699573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcacb-a380-48d5-b7dc-eae137cc43c4 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.701938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.702759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.703041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.703362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.705937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.707443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.709125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 97/192] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.713901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.745802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.746827 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.747114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.747408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.754835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.759718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:51.770563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:51.772071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 96/193] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:51.773178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.830829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dbdc7-654d-469e-ae92-8afad9e3eae3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.865402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.866066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.866346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.866619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.867394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:51.867699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.869077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.876978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 98/194] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.877855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.887106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9ab05-6997-4492-b533-7f0d35fa7049 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.896563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:51.921943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:51.922851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:51.923148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:51.923493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:51.936615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.938089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.940086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 97/195] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:51.943242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:51.980523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8895aa7-c852-4ddb-a52e-57a25b1a431a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:51.997595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.013978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.014650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.014938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.015261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.016260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.018464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.041395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bb6f8-addf-469c-b03b-370eeccb1e7a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.073365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.074021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.074303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.074575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.075338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:52.075646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.077013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.085394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 98/196] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:52.086181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.122220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 99/197] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:51 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:52.123171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.210469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c0a9c-6017-4f3b-bef9-d9e3838345f5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.211945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.230594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.239649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6453c00-6d22-40be-8e43-101c5fdee226 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.249701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.250601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.251414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.251705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.253706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.255745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.261971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 99/198] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:52.261971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.283608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.284328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.284615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.284904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.369502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.371662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.388600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0647d-6341-4506-bbc5-5787a3377d91 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.432518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.433412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.433703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.433982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] PATH_INFO: `/v3/users/0724864cb10741be9a9549f091020c1c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.435348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] RBAC: Authorizing `identity:get_user(user_id=0724864cb10741be9a9549f091020c1c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.436825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.439441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 100/199] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/users/0724864cb10741be9a9549f091020c1c => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:52.440310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.468568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.499729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 100/200] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:52.500597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.511539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.567329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24b0c-469e-4978-ad3f-af19aa235d34 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.609392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138b01-152d-4baa-92a8-da68d6fd1cf9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.610740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.611703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.612858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.613155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.648066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.648734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.649122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.649468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.701691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.705844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.708274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.728825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.730408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.783458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.835866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 101/201] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:52.836988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.847444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 101/202] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:52.848229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.927893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:52.943370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500a18-09a9-442b-bc39-2678c5acfcef None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.956017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4b42-02e2-4c28-b486-a6a2e3d37ed2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:52.983533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.984835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.985357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.986364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.987880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.989639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:52.991694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 102/203] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:52.992776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:52.995521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:52.996396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:52.996682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:52.997217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:52.998597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:52.999945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:53.002026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 102/204] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:53.002808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:53.021857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:53.112283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f4313-672e-4560-a71d-7d4df1438e3c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:53.113207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e30e42-eb60-48d9-83b9-3fd0e16afc60 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:53.147664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:53.148346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:53.148618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:53.148898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:53.149737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=faf005b3e2a24ea8a67866a2e4c76870) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:53.150058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:53.151262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:53.151522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:53.152537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:53.153015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:53.153542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:53.155330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:53.157546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:53.158270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 103/205] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:53 2022] GET /identity/v3/role_assignments?user.id=faf005b3e2a24ea8a67866a2e4c76870&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:53.159168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:53.160146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 103/206] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:52 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:53.171802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:53.173950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:53.270698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b49c9-304c-45f8-9d71-152e40b7c700 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:53.310387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:53.311102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:53.313458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:53.313960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:53.314872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=faf005b3e2a24ea8a67866a2e4c76870) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:53.315225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:53.316736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce4460-d9f0-46b6-a35a-a37483cb3eed None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:53.333790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 104/207] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:53 2022] GET /identity/v3/role_assignments?user.id=faf005b3e2a24ea8a67866a2e4c76870&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=d5708cbdc7894157a71389bd8517df5f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:53.358100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:53.367842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540be8e-fb29-4af5-aebd-a64b9d2a5a0c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:53.434987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:53.435619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:53.439618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:53.439903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:53.440821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2, user.id=0724864cb10741be9a9549f091020c1c) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:53.441144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:53.442705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576bfd9-9178-4150-b924-3fb6c7b0108c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:53.449815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 104/208] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:53 2022] GET /identity/v3/role_assignments?user.id=0724864cb10741be9a9549f091020c1c&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:54.394088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e1c9-e5d1-4ed5-a03b-b3eb366acc0c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.394628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e1c9-e5d1-4ed5-a03b-b3eb366acc0c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.394628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e1c9-e5d1-4ed5-a03b-b3eb366acc0c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.396232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 105/209] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:54.401725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cacbd-7be9-46b6-a8b2-7d200a5c3119 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.402540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cacbd-7be9-46b6-a8b2-7d200a5c3119 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.403028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cacbd-7be9-46b6-a8b2-7d200a5c3119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.421396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a65cacbd-7be9-46b6-a8b2-7d200a5c3119 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:54.422870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a65cacbd-7be9-46b6-a8b2-7d200a5c3119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.491760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 105/210] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:54.514839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786e1c9-e5d1-4ed5-a03b-b3eb366acc0c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.542065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f01ff8-f3af-4df2-a399-a7940717acfd None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.542361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f01ff8-f3af-4df2-a399-a7940717acfd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.542624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f01ff8-f3af-4df2-a399-a7940717acfd None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.544320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 106/211] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:54.549971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.550261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.550538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.567293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:54.568494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.599331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.642968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786e1c9-e5d1-4ed5-a03b-b3eb366acc0c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.647190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 107/212] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:57:54.665573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.681947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.684680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.685653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.686233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.686518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.701142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:54.712715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:54.713585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:54.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 106/213] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:54.722390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.778929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49a5d4-7c03-4e54-a25f-4e4d77d22f25 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.820235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.821075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.821361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.821640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.823213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.825555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.828162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 108/214] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:54.834740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.837016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296d4d-40e4-4d90-8977-62d5936be4d4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.868422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.874565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.875639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.876125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.876653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.877600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:54.883288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.883288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.895320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 107/215] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:54.897422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.924260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.943625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c37b33-b27f-4961-8829-d586c99f3b74 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.981468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.982430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.982558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.982805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.985188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.986422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.011192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d232e9-8787-495b-8a70-71f105ad26d8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.048013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.049416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.050007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.050517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] PATH_INFO: `/v3/users/c400878600604842ba5cb00977786677` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.059686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 109/216] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:54 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:55.067793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] RBAC: Authorizing `identity:get_user(user_id=c400878600604842ba5cb00977786677)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.069647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcb805-1f92-4d6f-bb99-3c15fa56b7d0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.076736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 108/217] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/users/c400878600604842ba5cb00977786677 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:55.078645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.199433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83298abd-3780-4439-870a-b78184a05d21 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.243056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.243761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.244050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.244331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.254460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.320379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.321806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.413916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0498b-7030-4262-aac6-e3d9d4bde862 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.414478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0498b-7030-4262-aac6-e3d9d4bde862 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.414478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0498b-7030-4262-aac6-e3d9d4bde862 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.416037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 109/218] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:55.421961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.422252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.422570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.445681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:55.446509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 110/219] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:57:55.446925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:55.452165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.501466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.514632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.547281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 110/220] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:55.574051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.593425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c2bd-40d5-451b-95c1-019de676522e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.640868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.642750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.643284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.643749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.645408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.647149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55ea5-c545-4e36-bd3b-ec9b294b9a25 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.649623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 111/221] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:55.651199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9d3c6-84a2-43c7-b9d3-c50b79d06a1a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.651731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9d3c6-84a2-43c7-b9d3-c50b79d06a1a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.652197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9d3c6-84a2-43c7-b9d3-c50b79d06a1a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.654032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 112/222] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:55.655299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9d3c6-84a2-43c7-b9d3-c50b79d06a1a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.717035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e342cb3-37d2-4ed5-a2f7-2bbce42d853a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.763312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.764399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.764695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.764976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.781086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f56b59d7-6db0-4051-af9a-c97f9746cbff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:55.789292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:55.790383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:55.791407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:55.792344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9d3c6-84a2-43c7-b9d3-c50b79d06a1a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.793519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 111/223] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:55.794954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.796073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.796073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.804310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.814705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:55.816065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:55.843403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.845371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.846115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.846590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.848042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=c400878600604842ba5cb00977786677) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:55.849094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.850813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.858663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 113/224] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/role_assignments?user.id=c400878600604842ba5cb00977786677&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:55.859623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.862139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.917276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 112/225] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:55.946182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.051827 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620e2b-0aa1-4c5b-90d9-d4afbc2c24be None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.080934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99f1e4-5ec4-4a55-94b1-ae5bd1db4a9f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.092201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.092885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.093175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.093510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.094222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:56.095147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.097503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.106530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 114/226] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:56.113645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.126449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.127535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.127986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.128408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.140998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:56.150420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:56.151498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:57:56.152466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:56.153178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 113/227] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:56.160066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.199865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.217153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.284208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb422-4b3d-40e0-b34a-1a1d5154b275 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.323073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.324802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.325242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.325692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.326599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:56.327061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.329200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.343387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 114/228] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:56.346858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.413410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a600-c3a9-48e9-862a-2380c4e724b9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.458635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c28de-0329-4f20-83c1-d615c51e6d99 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.462692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.470009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.470964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.471287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.471568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.475464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.476970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.478946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 115/229] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:56.484399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.498291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.499481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.499910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.500402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.502459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.504148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.506252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 115/230] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:56.511832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.651572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.655969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2b41-19c6-4342-bf80-003a95d52c42 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.680476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da39cc1-3638-41fc-83ea-92617e4dcd19 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.715947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.717440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.717741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.718027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.719512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.721457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.723601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 116/231] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:56.728923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.755814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.756742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.757031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.757311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.759841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.761321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.763424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 116/232] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:56.769694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.818974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.846067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d71f2-4d7a-4d46-873b-a5e9b7a31875 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.888519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.889425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.889713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.889995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/faf005b3e2a24ea8a67866a2e4c76870/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.892757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, user_id=faf005b3e2a24ea8a67866a2e4c76870, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.895388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb399de-8299-4896-8ece-e0a9d9e3a7fa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.929751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5bc74-69b5-4681-a33c-08f33e586f8d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.965979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.966952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.975736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.975736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/faf005b3e2a24ea8a67866a2e4c76870/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.975736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, user_id=faf005b3e2a24ea8a67866a2e4c76870, role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.975736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643a6e9-cc3f-4443-b6a6-4e7619fb11af None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.992725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 117/233] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:56 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/faf005b3e2a24ea8a67866a2e4c76870/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:57.061077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 117/234] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:56 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/faf005b3e2a24ea8a67866a2e4c76870/roles/d5708cbdc7894157a71389bd8517df5f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:57.670757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe189a-8a9e-4150-8943-7159a5ba992f None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.671634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe189a-8a9e-4150-8943-7159a5ba992f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.672054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe189a-8a9e-4150-8943-7159a5ba992f None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.674613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 118/235] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:57.680379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207ff2-041d-4f4e-afc3-030f1c452d9b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.681215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207ff2-041d-4f4e-afc3-030f1c452d9b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.681711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207ff2-041d-4f4e-afc3-030f1c452d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.698984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a8207ff2-041d-4f4e-afc3-030f1c452d9b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:57.700498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a8207ff2-041d-4f4e-afc3-030f1c452d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:57.779229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 118/236] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:57.784594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8fb5a-4d89-4c01-9ef2-2b02d2ae2c0e None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.785145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8fb5a-4d89-4c01-9ef2-2b02d2ae2c0e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.785616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8fb5a-4d89-4c01-9ef2-2b02d2ae2c0e None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.787216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 119/237] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:57.796559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.797127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.798274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.812144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8fb5a-4d89-4c01-9ef2-2b02d2ae2c0e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:57.835686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:57.836899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:57.864063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:57.870318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:57.959597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 119/238] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:57.983203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.019819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8fb5a-4d89-4c01-9ef2-2b02d2ae2c0e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.093646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74b392-1173-488d-afb6-180e3e701fd3 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.105223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.106034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.106316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.106587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.111883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.113383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.115066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 120/239] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:58.120633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.141493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.142478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.142766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.143043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.164013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:58.171744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:58.173258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:57:58.174112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:57:58.175304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 120/240] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:58.186837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.208313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.295171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.304039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862013f9-385d-4a09-8492-74c74e43d39a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.342434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f567d-b1ca-40c2-b5d2-9201c1ef9472 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.350844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.351573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.351862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.352191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.353008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.354553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91e98f-58ad-4fa1-b505-7f61c4616c84 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.393882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.394565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.394854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.395159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.395971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:58.396430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.398681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b8a7-ef9f-4dfd-9af4-41c08c2d3681 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.408280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 121/241] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:58.411239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1556dc-d7e1-4686-b513-1cb911f276cb None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.411239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1556dc-d7e1-4686-b513-1cb911f276cb None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.411239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1556dc-d7e1-4686-b513-1cb911f276cb None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.411680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 122/242] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:58.415316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1556dc-d7e1-4686-b513-1cb911f276cb None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.424354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 121/243] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:58.426034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.426541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.427046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.449061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:58.451345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.575519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.582620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 122/244] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:58.619852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.651994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1556dc-d7e1-4686-b513-1cb911f276cb None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.716826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.717708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.717989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.718345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] PATH_INFO: `/v3/users/cb508795f074443d9bd85e99e9eb4019` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.731063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.741425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cb508795f074443d9bd85e99e9eb4019)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.743697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.746652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 123/245] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/users/cb508795f074443d9bd85e99e9eb4019 => generated 297 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:58.753149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.756913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18601b-5aba-4f34-8d6f-4d826cefe524 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.798622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.799303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.799702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.800054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.816084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:58.826120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:58.826961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:57:58.827785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:57:58.830890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 123/246] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:57:58.837205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.892186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fdb51-dcb5-48d6-9f5d-61352d9064f6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.928071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.942061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.944084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.944576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.945048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.968990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343480-28c5-4859-8f8b-8058f1e5d434 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.014684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.015486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.016990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.017510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.018847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:59.019625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.021324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.033686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.034746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 124/247] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:59.037615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.043406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.047956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.169629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 124/248] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:57:59.175544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.192844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b5d28-8882-4d74-b06c-d3b35756b79d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.214884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.241643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.242778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.243250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.243760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] PATH_INFO: `/v3/users/c400878600604842ba5cb00977786677` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.245182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] RBAC: Authorizing `identity:get_user(user_id=c400878600604842ba5cb00977786677)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.246789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.248936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 125/249] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/users/c400878600604842ba5cb00977786677 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:59.257769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.320043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff41c-881a-425b-8f4c-65bc2690ac33 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.337709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.357807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.358974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.359438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.360511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.362021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.363728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.365619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 125/250] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:59.371718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.376388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7e4f-fd6a-4824-bfaa-eebdadc366e4 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.416572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.417864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.418354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.418862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.420438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.422110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.425173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 126/251] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:59.433299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.504893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.511827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd702fa-4046-4016-b697-c726217c054b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.551322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.551997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.552286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.552691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.553542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=cb508795f074443d9bd85e99e9eb4019) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:59.553861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.555352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b323a96-c095-4c99-a596-8c7ed57e0da7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.561933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 126/252] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/role_assignments?user.id=cb508795f074443d9bd85e99e9eb4019&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:59.563614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8a9d-db50-4069-8c73-fd9716f72350 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.563937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8a9d-db50-4069-8c73-fd9716f72350 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.564189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8a9d-db50-4069-8c73-fd9716f72350 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.565763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 127/253] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:59.566311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29c7c7-e022-43f7-8e3b-9d999ededd05 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.573135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d9641-3ebe-483b-9c03-acd2db385ec1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.573467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d9641-3ebe-483b-9c03-acd2db385ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.573712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d9641-3ebe-483b-9c03-acd2db385ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.604303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d66d9641-3ebe-483b-9c03-acd2db385ec1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.605546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d66d9641-3ebe-483b-9c03-acd2db385ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.640104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.641102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.641393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.641672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/c400878600604842ba5cb00977786677/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.645049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, user_id=c400878600604842ba5cb00977786677, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.653973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.735945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.741003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 127/254] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:59 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/c400878600604842ba5cb00977786677/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:59.758932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 128/255] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:57:59.786628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.871216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf96-1a8e-4b08-9058-5ccc89e88f18 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.871544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf96-1a8e-4b08-9058-5ccc89e88f18 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.872024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabaf96-1a8e-4b08-9058-5ccc89e88f18 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.874363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 129/256] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:59.880418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.880702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.880987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.914087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.920193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.957524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffa799-2433-4f88-9df3-599b05404da4 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.036295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.037028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.037319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.037600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.039458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 130/257] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:00.056061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:00.063900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:00.064772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:00.065569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 128/258] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:00.072976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.101804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:00.112970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.203163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:00.225573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812e962-d1ff-4f3c-b79c-0c270c0d7aef None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.285757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.286849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.287374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.287863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.288814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:00.291039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.292856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.311783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 131/259] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:00.314979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146085cf-8823-44ea-b1ca-63ccad3d2a34 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.336742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.396374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.397196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.397476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.397752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.431785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:00.444552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:00.445457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:00.446210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:00.448087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 129/260] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:00.458671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.533571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:00.589565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53e49-a156-4d72-b1c8-406173fe140c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.643066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b585c78-e0ce-4f62-8671-bf3e11da587f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.691280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:00.701161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.702191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.702674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.703173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.704809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:00.705335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.706933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.719285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.719285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.719285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.719285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.722529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 130/261] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:00.723941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.725396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.727081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 132/262] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:00.727882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.748167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.868625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbdc18-0006-4cfa-8ac9-79ed859ba5a9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.916594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.917825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.918332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.918818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] PATH_INFO: `/v3/users/faf005b3e2a24ea8a67866a2e4c76870` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.920287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] RBAC: Authorizing `identity:get_user(user_id=faf005b3e2a24ea8a67866a2e4c76870)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.921938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.924696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 131/263] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/users/faf005b3e2a24ea8a67866a2e4c76870 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:00.929796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:01.010422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.031270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405e0a-115a-4aa3-95f8-e343e891a589 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:01.075830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:01.076690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.076911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.077155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.103500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.149289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95481d0e-e916-459c-947d-3a0170a87616 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:01.166713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:01.168424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:01.228967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:01.229659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.229950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.230230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.343070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 133/264] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:01.352339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:01.384011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:01.386626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:01.463559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.515630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.542933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67f60-0817-421e-b3d7-c9966c9192b8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:01.604864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 132/265] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:01.611981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:01.640456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:01.641869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.642161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.642433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.643840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:01.645300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:01.647366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 134/266] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:01 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:01.652974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:01.800096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f7297-940f-47d8-866e-290f9c492e33 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:01.858554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:01.859506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.859838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.860128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.861343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:01.862811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:01.864876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 133/267] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:01 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:01.870626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:01.896868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.906203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22005e8-c74c-4eb7-92e7-0d0d9c0f3e20 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:01.944496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:01.971000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:01.971934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.972723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.973464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.974455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=faf005b3e2a24ea8a67866a2e4c76870) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:01.974917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:01.976552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578143e7-874a-4300-ba22-21e30b14ee73 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:01.983668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 135/268] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:01 2022] GET /identity/v3/role_assignments?user.id=faf005b3e2a24ea8a67866a2e4c76870&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:01.985935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2951-6258-4d00-941f-3f50602e3288 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.986374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2951-6258-4d00-941f-3f50602e3288 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.986796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b2951-6258-4d00-941f-3f50602e3288 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:01.988888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 136/269] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:01.995140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab17c-1d9f-4bc8-b51c-d08fbbcf252c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:01.995678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab17c-1d9f-4bc8-b51c-d08fbbcf252c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:01.997222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab17c-1d9f-4bc8-b51c-d08fbbcf252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.030808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-eb1ab17c-1d9f-4bc8-b51c-d08fbbcf252c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:02.032587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-eb1ab17c-1d9f-4bc8-b51c-d08fbbcf252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:02.063489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bb164-f8e3-4860-89cd-ddcdc10ea3ad None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.134978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.136053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.136541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.137238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.138840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=faf005b3e2a24ea8a67866a2e4c76870) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:02.139424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:02.141761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:02.158215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 134/270] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:01 2022] GET /identity/v3/role_assignments?user.id=faf005b3e2a24ea8a67866a2e4c76870&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=d5708cbdc7894157a71389bd8517df5f => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:02.214131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 137/271] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:02.254194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.349985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629c30-1349-48bb-b5f8-58c860ea102e None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.350275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629c30-1349-48bb-b5f8-58c860ea102e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.350477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05629c30-1349-48bb-b5f8-58c860ea102e None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.352140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 138/272] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:02.358861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387fa-1791-47cf-9df6-5fe10aeb4663 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.359192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387fa-1791-47cf-9df6-5fe10aeb4663 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.360006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387fa-1791-47cf-9df6-5fe10aeb4663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.366644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:02.395620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22354ae-0322-4b66-9ced-c6f8bf69653c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.398202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ad7387fa-1791-47cf-9df6-5fe10aeb4663 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:02.399457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ad7387fa-1791-47cf-9df6-5fe10aeb4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:02.462670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.464025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.464511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.465022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.481945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:02.485461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 139/273] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:02.486628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31c1c-bd65-4aab-b66a-e1d2e941c55b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.486921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31c1c-bd65-4aab-b66a-e1d2e941c55b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.487344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce31c1c-bd65-4aab-b66a-e1d2e941c55b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.488890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 140/274] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:02.491778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:02.492663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:02.493422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:02.495926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.496217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.496506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.497012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 135/275] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:02.502325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.515550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:02.517277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:02.567087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:02.605376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 141/276] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:02.607155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.626494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be232ce5-d0f8-4491-abb7-67baf122ca94 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.672335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.673017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.673302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.673580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.674336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:02.674904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:02.676095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:02.684547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 136/277] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:02.685348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.695583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:02.726617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcabf69-4066-421a-af9d-07fe241917b1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.763865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.764278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.764513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.764784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.767165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:02.768993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:02.770447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 142/278] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:02.775416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.804488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d83e-360f-4294-8fc9-d00cfa31042e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.820856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:02.841939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.842740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.843028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.843424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.855331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:02.863374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:02.864320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:02.865122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 137/279] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:02.865775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.883720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b46c20-beda-4c88-8788-8afc8fabe011 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:02.918424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:02.919332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.919623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.919896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] PATH_INFO: `/v3/users/09903aa4b3494da8b0a29e4e95b4255a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.926716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:02.933418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] RBAC: Authorizing `identity:get_user(user_id=09903aa4b3494da8b0a29e4e95b4255a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:02.938806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709317e0-4cb7-4f53-9527-59662c8c1e28 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:02.938806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 143/280] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/users/09903aa4b3494da8b0a29e4e95b4255a => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:02.938806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca57-b505-4cd8-8ebe-ed5dbeed4535 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:02.938806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca57-b505-4cd8-8ebe-ed5dbeed4535 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:02.939299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca57-b505-4cd8-8ebe-ed5dbeed4535 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:02.940580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 144/281] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:02.941145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9ca57-b505-4cd8-8ebe-ed5dbeed4535 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:02.970226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbebc4b-ec92-4481-8862-c7e6703efcbc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.004598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.005267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.005550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.005828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.006534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.008027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.038521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.043191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9ca57-b505-4cd8-8ebe-ed5dbeed4535 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.077138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.077807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.078144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.078585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.079370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:03.079675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.081053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd04a-53bf-41c2-81b3-6234ea8916e2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.082029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 138/282] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:02 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:03.082863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.090608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 145/283] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.091769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.092075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.092455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.113806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:03.114942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:03.215148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 146/284] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:03.215148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.215404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6d08b-51e5-4bcf-81b3-1abd9909d36f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.243030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.262653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.263731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.264015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.264292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.265127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, user.id=09903aa4b3494da8b0a29e4e95b4255a, scope.system=all) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:03.265534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.267680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.273857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 139/285] 10.222.0.59 () {56 vars in 1308 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/role_assignments?user.id=09903aa4b3494da8b0a29e4e95b4255a&scope.system=all&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.274497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.321118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.339824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa6416-2137-4d83-9e93-6c762aab2c05 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.388694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.389669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.389959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.390249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] PATH_INFO: `/v3/users/cb508795f074443d9bd85e99e9eb4019` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.392130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] RBAC: Authorizing `identity:get_user(user_id=cb508795f074443d9bd85e99e9eb4019)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.393740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.395876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 147/286] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/users/cb508795f074443d9bd85e99e9eb4019 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.401277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.447042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781750e-8eb5-417e-af90-167e308dfdfc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.489770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.490961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.491473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.491942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.506752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:03.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:03.517963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:03.518732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:03.521401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 140/287] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:03.526114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18dd76-df48-4240-bda8-ff22ce6fd009 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.527299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.586771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.599918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.600791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.601071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.601707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.603094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.605692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.606862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 148/288] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.616124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.652345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d62fd-401e-4be1-9608-979235b058a0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.694886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.696335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.696622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.696900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.697652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:03.697962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.699381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.708703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 141/289] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.710363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.715164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:03.842745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ae12-90d4-4dd2-b746-1f6aa1408895 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.876713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3913c-773b-4c39-af3a-6f8264d0466f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:03.894254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:03.920439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.921669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.922666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.923166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/cb508795f074443d9bd85e99e9eb4019/roles/a5df04a57f184f42ba005fa9eb51795c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.926633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, user_id=cb508795f074443d9bd85e99e9eb4019, role_id=a5df04a57f184f42ba005fa9eb51795c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.928344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.946662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:03.948223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:03.948851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:03.949319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] PATH_INFO: `/v3/users/c400878600604842ba5cb00977786677` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:03.951155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] RBAC: Authorizing `identity:get_user(user_id=c400878600604842ba5cb00977786677)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:03.952780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:03.954957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 142/290] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/users/c400878600604842ba5cb00977786677 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:03.960059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.023293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 149/291] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:03 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/users/cb508795f074443d9bd85e99e9eb4019/roles/a5df04a57f184f42ba005fa9eb51795c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:04.079736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9601623-eb16-471d-9ce0-e2acd885e217 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.128224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.129222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.129699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.130149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.199375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:04.235592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.237107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.372198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 143/292] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:04.378103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.472327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:04.488735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e016df-db43-491e-80ad-7ece0613866f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.526064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.526956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.527272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.527553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.528773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.530500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc192da-28a6-4ba1-9ac1-e7cd63e00bb1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.532772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 150/293] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:04 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:04.538483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.623926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:04.656909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032b272-bf45-4477-a4a3-ee393de40043 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.710051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.710737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.711025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.711330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.712187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=c400878600604842ba5cb00977786677) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:04.712505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.713975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49734eff-1239-4088-90cb-a51fafe23d88 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.724564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 144/294] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:04 2022] GET /identity/v3/role_assignments?user.id=c400878600604842ba5cb00977786677&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:05.538068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2125776-d0fb-4d99-b3d8-7aba19e4d1e1 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.538586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2125776-d0fb-4d99-b3d8-7aba19e4d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.538586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2125776-d0fb-4d99-b3d8-7aba19e4d1e1 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.540277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 151/295] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:05.545511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.545897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.546076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.564287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:05.565478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:05.602461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7179a7-77a3-46aa-98e2-ca66d808fb9b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.603072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7179a7-77a3-46aa-98e2-ca66d808fb9b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.603660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7179a7-77a3-46aa-98e2-ca66d808fb9b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.605492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 152/296] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:05.611607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.612143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.612667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.631014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:05.632619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:05.637124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 145/297] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:05.661114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.708069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 153/298] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:05.730955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:05.733001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.766507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9cac-f1b9-47b2-8cdf-f1de544a2411 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.801472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.802194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.802475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.802748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.813655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:05.821290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:05.822170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:05.822950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 146/299] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:05.829611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.832090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:05.845437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849a42c-1a62-4644-9b4d-877d20265510 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.887592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.888556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.889161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.889447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.900962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a4c3c134-f593-4d6c-bf46-aef24f388140 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:05.909026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:05.909884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:05.911087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 154/300] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:05.913384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49607ae1-a915-4e0e-916f-ed3e14ddd45f None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.913676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49607ae1-a915-4e0e-916f-ed3e14ddd45f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.913960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49607ae1-a915-4e0e-916f-ed3e14ddd45f None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.916359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 155/301] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:05.916359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49607ae1-a915-4e0e-916f-ed3e14ddd45f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.942843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a894-d3c3-4482-ad4b-4aef3119fd79 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.978197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:05.987754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.988469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.989217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.989506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.990543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:05.995964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:05.995964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503697a-78ff-4602-a2a0-9bc1317b27f9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.008678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 147/302] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:06.008678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.008678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.008678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.027017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:06.027210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49607ae1-a915-4e0e-916f-ed3e14ddd45f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.028304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:06.064299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.064897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.065187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.065466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.066218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:06.066530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.067943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.080963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 156/303] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:06.080963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.088667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.104246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 148/304] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:06.105032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.203606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3476758-60d8-45f9-bfd0-73f8a34d7eec None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.216542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5381-7a2b-4a9d-bd50-04f52d9368f3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.239074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.246091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.247315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.248171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] PATH_INFO: `/v3/users/83c8c623b09b46d998365915e1e7b42d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.255992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.256881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.257170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.257453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] PATH_INFO: `/v3/users/09903aa4b3494da8b0a29e4e95b4255a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.258693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] RBAC: Authorizing `identity:get_user(user_id=09903aa4b3494da8b0a29e4e95b4255a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.260191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.266621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 149/305] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/users/09903aa4b3494da8b0a29e4e95b4255a => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:06.266621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.271155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] RBAC: Authorizing `identity:get_user(user_id=83c8c623b09b46d998365915e1e7b42d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.272654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.279621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 157/306] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/users/83c8c623b09b46d998365915e1e7b42d => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:06.279621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.356365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.375257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0940-c91c-49ec-9aec-d572032e377f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.387654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77d84-ace6-47d2-af87-0b5418a3fdb5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.410559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.411393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.411678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.411952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.413123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.414585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.416225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 150/307] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:06.421530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.425537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.426415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.426704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.426978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] PATH_INFO: `/v3/system/users/09903aa4b3494da8b0a29e4e95b4255a/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.428593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=09903aa4b3494da8b0a29e4e95b4255a, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.429981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.478474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.499642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 158/308] 10.222.0.59 () {58 vars in 1334 bytes} [Thu Jul 14 10:58:06 2022] PUT /identity/v3/system/users/09903aa4b3494da8b0a29e4e95b4255a/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:06.500483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.530103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b8aa-2f1b-4672-98ea-76a189559472 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.569880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.570566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.570837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.571139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.572001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, user.id=83c8c623b09b46d998365915e1e7b42d, scope.system=all) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:06.572323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.573860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b5258-74fe-44a6-863c-eb416e70c28c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.591757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 151/309] 10.222.0.59 () {56 vars in 1308 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/role_assignments?user.id=83c8c623b09b46d998365915e1e7b42d&scope.system=all&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:06.594308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26579-3c58-452a-be13-8ae4ec4f86dc None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.594308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26579-3c58-452a-be13-8ae4ec4f86dc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.594308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26579-3c58-452a-be13-8ae4ec4f86dc None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.596468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 152/310] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:06.608765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.608961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.609402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.630742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:06.632364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:06.642991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7bba2-9566-45a5-ae18-574a06582adc None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.688155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.698925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.700152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.704408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.704408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] PATH_INFO: `/v3/groups` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.704408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.704408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.724574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 153/311] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:06.748390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.782254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 159/312] 10.222.0.59 () {60 vars in 1134 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/groups => generated 229 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:06.809077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.869712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf53d5f-fbf6-46e7-b802-218e24aa0176 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.909130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.909948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.910231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.910506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.923144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:06.931205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:06.932175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:06.932982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 154/313] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:06.939088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.053827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8148520-a4da-41da-8d47-9b2495e5e35b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.091875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.092555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.092883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.093163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.094683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:07.094992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.096445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.107862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 160/314] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:07.114569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.222901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb0a4-440c-4ad5-b385-69ac68ec6ff3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.240606 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:07.258443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.259355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.259637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.264910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] PATH_INFO: `/v3/users/cb508795f074443d9bd85e99e9eb4019` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.264910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] RBAC: Authorizing `identity:get_user(user_id=cb508795f074443d9bd85e99e9eb4019)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.264910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.264910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 155/315] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:07 2022] GET /identity/v3/users/cb508795f074443d9bd85e99e9eb4019 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:07.270168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.323810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:07.379462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995bef-f38e-44ad-b566-3bba9f2917c5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.414922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.415879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.416362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.416821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.488261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.489982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.543562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:07.595907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 161/316] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:07.602407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.709921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7389cb-cd85-4b4f-8e9c-10488f44f359 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.745617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.746841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.747362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.747852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.749220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.751499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ff083-8b0a-4aeb-9c11-0c1d6978462a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.753712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 156/317] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:07 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:07.759942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.864901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199d6a-0193-4448-a22f-63611ad8af67 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.899043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.899727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.900014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.900297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.901128 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d, user.id=cb508795f074443d9bd85e99e9eb4019) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:07.901442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.902867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b08f4-85c6-4e3a-bb91-3729fb61584e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.910360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 162/318] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:07 2022] GET /identity/v3/role_assignments?user.id=cb508795f074443d9bd85e99e9eb4019&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:08.642450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731c8f-5088-43c7-8f22-7501fb99f3af None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:08.643478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731c8f-5088-43c7-8f22-7501fb99f3af None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:08.644045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f731c8f-5088-43c7-8f22-7501fb99f3af None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:08.645891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 157/319] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:08.650812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:08.651098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:08.651413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:08.668734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:08.669924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:08.681529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:08.732960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923d4c-2840-4a5b-92b5-9b20943235be None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:08.733249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923d4c-2840-4a5b-92b5-9b20943235be None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:08.733528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923d4c-2840-4a5b-92b5-9b20943235be None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:08.735785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 158/320] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:08.740668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 163/321] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:08.740795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:08.740895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:08.741143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:08.758614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:08.759826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:08.764112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:08.806160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:08.831509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 159/322] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:08.853467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:08.870729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd5ee6-36b6-4421-afc8-09d7067fe978 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:08.910752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:08.911512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:08.911795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:08.912064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:08.923502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:08.931367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:08.932251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:08.933009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 164/323] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:08.939210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:08.942862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:08.964799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8205b9d-5ef3-4fcd-8613-160fffe51f2d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.002466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.003293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.003577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.003854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.013746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-2763707f-aecd-4f60-93c0-47e9571e4d59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:09.021471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:09.022405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:09.023217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 160/324] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:09.024087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5eced-0a3e-40a3-9a56-d0ce3a665a33 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.024221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5eced-0a3e-40a3-9a56-d0ce3a665a33 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.024539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5eced-0a3e-40a3-9a56-d0ce3a665a33 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.025740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 161/325] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:09.028247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5eced-0a3e-40a3-9a56-d0ce3a665a33 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.049454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3beeaf-f71a-4ec3-b341-8a11c34d1fd6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.064904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.082716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.083418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.083706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.083989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.084727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:09.085038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.086576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fa5bd-c68e-4a60-a28f-c8e77f0c68e3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.094616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 165/326] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:09.095916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.097043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.097362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.113910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:09.115134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:09.131570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5eced-0a3e-40a3-9a56-d0ce3a665a33 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.165091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.165707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.165986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.166258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.167030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:09.167370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.168809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.174649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.179218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 162/327] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:09.190390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 166/328] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:09.190604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.190689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.284607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.288324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c1d7-02e0-460f-ba15-9238ecbf3faf None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.288443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c88c9-2578-44ee-82bb-963650fe5fdf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.322146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.322763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.323017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.323332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.323613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] PATH_INFO: `/v3/users/09903aa4b3494da8b0a29e4e95b4255a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.323786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.324133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.324413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] PATH_INFO: `/v3/users/83c8c623b09b46d998365915e1e7b42d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.324804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] RBAC: Authorizing `identity:get_user(user_id=09903aa4b3494da8b0a29e4e95b4255a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.325721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] RBAC: Authorizing `identity:get_user(user_id=83c8c623b09b46d998365915e1e7b42d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.326287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.327087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.332905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 163/329] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:09 2022] GET /identity/v3/users/09903aa4b3494da8b0a29e4e95b4255a => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:09.333060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 167/330] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:09 2022] GET /identity/v3/users/83c8c623b09b46d998365915e1e7b42d => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:09.333151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.334280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.395418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.432209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77ca3d-3e41-4075-90ab-5366bb8cd567 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.433963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a1dc-441f-4922-bbfe-337cc0c2f474 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.464878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.465894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.466178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.466274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.466457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.466895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.467205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.467480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.467647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.468850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] RBAC: Adding query filter params (role.id=f64ed86228d24c5b8f70e7a0e8625010, user.id=09903aa4b3494da8b0a29e4e95b4255a, scope.system=all) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:09.469408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.469596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.470773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.471258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 164/331] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:09.471997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.487213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 168/332] 10.222.0.59 () {56 vars in 1308 bytes} [Thu Jul 14 10:58:09 2022] GET /identity/v3/role_assignments?user.id=09903aa4b3494da8b0a29e4e95b4255a&scope.system=all&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:09.487989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:09.521099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.600046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2d794-fc8e-46c5-b0e3-57d3ed1f57fe None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.603739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442b38-780f-430a-a6e8-a73a4670a2c3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:09.632061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:09.638131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.639011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.639330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.639645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] PATH_INFO: `/v3/system/users/83c8c623b09b46d998365915e1e7b42d/roles/a5df04a57f184f42ba005fa9eb51795c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.642138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=83c8c623b09b46d998365915e1e7b42d, role_id=a5df04a57f184f42ba005fa9eb51795c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.643607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67c8b0-5361-4e48-9767-c3232977db8c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.648448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:09.648765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:09.649052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:09.649333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] PATH_INFO: `/v3/groups` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:09.650058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:09.651594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692f816-99af-4635-9bdd-b95d192f31b3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:09.745117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 165/333] 10.222.0.59 () {58 vars in 1334 bytes} [Thu Jul 14 10:58:09 2022] PUT /identity/v3/system/users/83c8c623b09b46d998365915e1e7b42d/roles/a5df04a57f184f42ba005fa9eb51795c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:09.761705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 169/334] 10.222.0.59 () {60 vars in 1134 bytes} [Thu Jul 14 10:58:09 2022] POST /identity/v3/groups => generated 226 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:11.921701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5259-ec47-4dff-b671-dd47ec2b6dcd None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:11.922221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5259-ec47-4dff-b671-dd47ec2b6dcd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:11.922334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f5259-ec47-4dff-b671-dd47ec2b6dcd None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:11.923957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 166/335] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:11.930064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:11.930499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:11.930727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:11.953213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:11.954531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.043048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 170/336] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:12.078934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f5259-ec47-4dff-b671-dd47ec2b6dcd None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.201850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.207349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f5259-ec47-4dff-b671-dd47ec2b6dcd None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.256787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.257602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.257890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.258169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.273478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f755713c-22fe-4ada-939d-45603f5c5a67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:12.282241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:12.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:12.284324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 167/337] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:12.289872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.318424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddc42e-1d44-414d-8033-b84d6ed9f15a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.318738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddc42e-1d44-414d-8033-b84d6ed9f15a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.318991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddc42e-1d44-414d-8033-b84d6ed9f15a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.320651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 168/338] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:12.327376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.328187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.328471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.355942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:12.357167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.486730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ae540-7d04-4f6a-8756-e2f6e3767a25 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.515493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.537849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 169/339] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:12.565282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.565967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.566254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.566540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.567406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:12.567719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:12.569106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:12.586765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 171/340] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:12.592192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.598695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.731647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.747425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6d04c-4ac2-439d-9d73-d04becdf37d7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.788558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74166b-7c88-4418-87c0-91cbaa554e44 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.797624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.798532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.798817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.799093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] PATH_INFO: `/v3/users/83c8c623b09b46d998365915e1e7b42d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.800400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] RBAC: Authorizing `identity:get_user(user_id=83c8c623b09b46d998365915e1e7b42d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:12.801882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ef699-29b3-438e-8349-c0c7216633a5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:12.803585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 170/341] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/users/83c8c623b09b46d998365915e1e7b42d => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:12.804857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7faf55-8ae5-4a7c-9eaa-e247a2c34ad7 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.805150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7faf55-8ae5-4a7c-9eaa-e247a2c34ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.805430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7faf55-8ae5-4a7c-9eaa-e247a2c34ad7 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.806657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 171/342] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:12.812576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7faf55-8ae5-4a7c-9eaa-e247a2c34ad7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.836854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.837648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.837929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.838199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.857465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-399660da-39af-4f22-81e2-f4596764f9b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:12.867748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:12.868630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:12.869861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 172/343] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:12.870716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.871069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.871663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.892981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:12.894182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.928018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.991450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 173/344] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:12.992322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.022610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7faf55-8ae5-4a7c-9eaa-e247a2c34ad7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.067035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.067764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.068046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.068322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.069163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] RBAC: Adding query filter params (role.id=a5df04a57f184f42ba005fa9eb51795c, user.id=83c8c623b09b46d998365915e1e7b42d, scope.system=all) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:13.069529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.071536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.090783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 172/345] 10.222.0.59 () {56 vars in 1308 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/role_assignments?user.id=83c8c623b09b46d998365915e1e7b42d&scope.system=all&role.id=a5df04a57f184f42ba005fa9eb51795c => generated 510 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:13.091683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.129609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.145954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2043c-92b7-443d-a249-6307bc607013 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.185903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.186773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.187215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.187559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.188329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:13.188635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.195152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.200275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 174/346] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:13.206224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.240952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.325584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e77e4-689c-482e-ab7c-8ddc47498a6c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.334662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370c40f-17e2-4d1f-8773-f1dde50d75b7 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.392299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.393230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.393516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.393790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.400180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.401010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.401295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.401572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.425723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b0cf253d-54f1-404f-9d57-61bab95dfca5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:13.434712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:13.435777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 173/347] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:13.438121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8455-4b72-43e7-98d3-3b067bf045c8 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.439914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8455-4b72-43e7-98d3-3b067bf045c8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.439914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba8455-4b72-43e7-98d3-3b067bf045c8 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.444155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 174/348] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:13.445452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba8455-4b72-43e7-98d3-3b067bf045c8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.482641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.485538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefe1f5-e67c-4ee8-989f-7a1837f81326 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.522052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.547348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.578774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba8455-4b72-43e7-98d3-3b067bf045c8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.617957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 175/349] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:13.619275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.619563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.619858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.620276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.620957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.621241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.621522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.622280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:13.622590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.624034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.634633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 175/350] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:13.635509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.643260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:13.644780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:13.741748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 176/351] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:13.742928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.764881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98390881-1e5a-4861-93ae-4396dd167582 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.802640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.803597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.803886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.804162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.806038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.807647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.809393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 176/352] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:13.810654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.815388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.821141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.879205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df4c3-1746-4863-8555-54c720d9c767 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.925585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.926377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.926658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.927200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.931923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2697b-238f-4b40-8653-7ce29c16d87b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.980841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.981659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.982305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.982655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.000628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:14.009751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:14.010661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:14.011516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:14.013058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.014558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cd498-c8c8-4aa4-97ad-c4f32a0689bc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.015767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 177/353] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:14.016850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.092032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.098161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.144407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395b9a9-209f-4b59-a500-a1eb61e1a682 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.148615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 177/354] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:14.150115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d144f-648e-4b82-a9fd-e48e9de4bee2 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.150540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d144f-648e-4b82-a9fd-e48e9de4bee2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.150819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d144f-648e-4b82-a9fd-e48e9de4bee2 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.152365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 178/355] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:14.153077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d144f-648e-4b82-a9fd-e48e9de4bee2 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.203556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.204439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.204725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.205004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.212739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.214211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d3403-a5b7-4dfb-a391-905627c3eeec None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.216476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 178/356] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:14.218495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.218785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.219416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.241550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:14.241550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:14.294109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d144f-648e-4b82-a9fd-e48e9de4bee2 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.331620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 179/357] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:14.332849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.335311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.336258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.336740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.337285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.342639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:14.342639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.342639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.349729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 179/358] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:14.350799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.378454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.415351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.465806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d50533-a7ef-4ebd-a46d-c2245da4abf1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.475612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05df5-68ad-4670-be05-5034a7f2e17a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.510571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.513282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.513282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.513282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.513282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.515272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.516990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 180/359] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:14.517770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.519852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.520916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.522234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.522722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.523614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:14.523929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.525365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.532773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 180/360] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:14.533590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.648627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.656389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb0a85-efcc-4cc7-b29a-c350aa1419aa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.672310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04daaa7-9ba6-4b0c-8c06-4ddb6b8c1e89 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.693355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.703901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.704561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.704857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.705142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.715667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.716748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.717197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.717614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.731183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-91900354-e07f-4cfd-a57d-25d6acd530f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:14.741836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:14.742940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:14.743877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 181/361] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:14.744529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e761-4c41-4878-8585-b5413a9dbc50 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.744635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e761-4c41-4878-8585-b5413a9dbc50 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.744922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e761-4c41-4878-8585-b5413a9dbc50 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.749678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 182/362] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:14.751380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e761-4c41-4878-8585-b5413a9dbc50 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.789091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.790619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.885638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e761-4c41-4878-8585-b5413a9dbc50 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.931616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 181/363] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:14.931616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.938457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.939398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.939689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.939971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.941175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.943078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac135-21b9-4af0-b5f6-07afdce8d002 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.949337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 183/364] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:14.949337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.949337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.949337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.963932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.972221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:14.974167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:14.977330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.059972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 184/365] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:15.060701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.065322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7377e-3c2b-4381-932c-5e60b865c807 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.104796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.106226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.106616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.106998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.107919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:15.108235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.110039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.118990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 182/366] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.119863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.173388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1838b-48cd-45d0-ae08-54f3dcd0e079 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.230140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.230140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.230140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.230140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.231366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.235828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.237046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 185/367] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.237180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.245621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.257780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.259950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fd8ce-3602-4bc8-bdf2-0350b34516c8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.299432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.300102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.300387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.300667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.301509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:15.301825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.303344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.311173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 183/368] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=d5708cbdc7894157a71389bd8517df5f => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.312726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.377979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52f01-e64d-4ca0-bb59-facd8a23c27d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.433170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.434247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.434534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.434815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.443780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bf7e4-fd42-4782-86a8-802cc6182f14 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.451533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:15.463361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:15.464312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:15.465085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:15.465669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 186/369] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:15.466963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.493847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.494429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.494712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.494987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.535457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.580045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.607866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.610786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.624647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232dbc3-528d-4095-8933-f938e4f4538f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.664360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.665532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.666007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.666473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.668613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.671957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.674271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 187/370] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.679908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.734538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 184/371] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:15.735534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.805983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.812487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16305933-b7fb-4f7c-8c59-894e2d236160 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.853657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.854785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.855333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.855823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.856905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:15.857500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.859623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.862626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.864777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed032ca6-84e7-42d5-a5b9-ad81d9d2e243 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.871463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 188/372] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.872648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.901407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.902068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.902350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.902622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.903747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:15.904059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.906337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.913083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 185/373] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:15.913892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.992423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac316a-8ac7-4708-9596-fd58b7980300 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.023769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd950e42-9606-4cbd-80e0-d518f6bbb261 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.032162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.033461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.033956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.034434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.036312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.038035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.040614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 189/374] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.042654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.046717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.068219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.068850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.069138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.069413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.143531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.156814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.158339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.184506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308faa18-7734-4ca4-8d0e-eaad1f65dd7c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.224855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.225858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.226236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.226527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.228947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.230433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.232483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 190/375] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.239436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.294812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 186/376] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:16.304363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.338594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.365642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f2f1-108f-440a-952a-c49b2dae002b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.406856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.407614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.407928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.408612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.409517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:16.409840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.411880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.413863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.418636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 191/377] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=d5708cbdc7894157a71389bd8517df5f => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.497191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529db1-410b-4950-b752-5726bd10f036 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.551331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.552213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.552495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.552772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] PATH_INFO: `/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.555815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] RBAC: Authorizing `identity:get_project(project_id=7da05e4cf2b444729a9cbe3b6d5db535)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.557300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.559475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 187/378] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535 => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.565590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.724255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80765-a757-4f0e-b8a3-5dd61365e806 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.768767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.769646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.769936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.770214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] PATH_INFO: `/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.779933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] RBAC: Authorizing `identity:get_group(group_id=3fb1dc7f3bc4468d9d794f04379fac9c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.782272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b04237-bcfd-4da8-a599-70b8a22c8ca3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.784414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 192/379] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.790149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.898962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.915450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6df134-077c-4acb-a5fa-81b1be0b3752 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.964876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.965540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.965819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.966092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.966934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] RBAC: Adding query filter params (group.id=3fb1dc7f3bc4468d9d794f04379fac9c, role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=7da05e4cf2b444729a9cbe3b6d5db535) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:16.967270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.968947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc12eb8-f2c8-4fe7-a32d-aa20cd7ed33a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.979566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 188/380] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/role_assignments?group.id=3fb1dc7f3bc4468d9d794f04379fac9c&scope.project.id=7da05e4cf2b444729a9cbe3b6d5db535&role.id=beb4a709a14b415db991ad1980b406a9 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:17.168496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108384e9-6c6a-4d99-ad5c-a50298070fc1 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.169026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108384e9-6c6a-4d99-ad5c-a50298070fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.169119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108384e9-6c6a-4d99-ad5c-a50298070fc1 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.171000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 193/381] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:17.178342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.178342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.178763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.205216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:17.206508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:17.295222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 189/382] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:17.335477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108384e9-6c6a-4d99-ad5c-a50298070fc1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.418817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:17.470565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108384e9-6c6a-4d99-ad5c-a50298070fc1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.511245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.512051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.512339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.512618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.529114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:17.537761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:17.538685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:17.539934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 194/383] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:17.548168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.664464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:17.689935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebed22d-34f5-433d-974f-6ecda2806b4a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.732758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.733652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.734085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.734368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.735138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:17.735447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.736818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.746861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 190/384] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:17.753195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.907004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778cf3c-ecff-4fe4-ae22-e4388595fff6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.941001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:17.953287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.954161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.954444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.954764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.959848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.961327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.964011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 195/385] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:17.969300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.119759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2eefa-6b5d-480b-b5b1-18fbb131fd22 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:18.164788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:18.165586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.165867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.166140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.167353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:18.168787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70245670-d743-43ac-9799-5808145fdbdc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:18.171418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 191/386] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:18.183169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.183602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47804-741f-4e44-89fe-99774a9d278d None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.183602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47804-741f-4e44-89fe-99774a9d278d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.183602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb47804-741f-4e44-89fe-99774a9d278d None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.183602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 192/387] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:18.192740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.193025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.193305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.224521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:18.225720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:18.231268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:18.360294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ae35-3f45-4af6-8e10-eaa71737b28a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:18.374842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 193/388] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:18.407073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.443039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:18.443874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.444161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.444441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/groups/42376ceca8464b31bceb2d35b76df506/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.449706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, group_id=42376ceca8464b31bceb2d35b76df506, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:18.451186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:18.533971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 196/389] 10.222.0.59 () {58 vars in 1442 bytes} [Thu Jul 14 10:58:18 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/groups/42376ceca8464b31bceb2d35b76df506/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:18.543718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2f888-35bc-4d25-af66-deb1f5bf554d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:18.584137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:18.585060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.585351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.585630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.594076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:18.598580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:18.607083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:18.611112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:18.612028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 194/390] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:18.615222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.680356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:18.752589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309bc74-64b0-47db-993a-89127e71c97b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:18.791287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:18.791966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.792251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.792530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.793285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:18.793595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:18.794964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab804734-88df-4e4c-9262-dc1544c3f9c8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:18.804339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 197/391] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:18.809917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.811909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bd22-a0c7-48cf-8891-37c8872c287b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.812210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bd22-a0c7-48cf-8891-37c8872c287b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.812489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bd22-a0c7-48cf-8891-37c8872c287b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.814145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 198/392] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:18.819393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.819685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.819966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.841524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:18.842788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:18.966912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 199/393] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:18.990473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2218e96-4a95-499a-9830-8078e593b59f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.015229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.056777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.058007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.058135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.058606 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.059087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.067917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.069678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.071640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 195/394] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:18 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:19.077033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.131536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd39af0-308e-41eb-a5ab-04a33ae4d9ab None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.166228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.168636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.169725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.170280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.171379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.184987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-2c9b91b5-852e-4e87-9386-052b875ec684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:19.194044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:19.194928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:19.195745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:19.198976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 200/395] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:19.200473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18008cc-6b72-4c49-a189-852915f4c727 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.200974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18008cc-6b72-4c49-a189-852915f4c727 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.201445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18008cc-6b72-4c49-a189-852915f4c727 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.203668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 201/396] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:19.205930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18008cc-6b72-4c49-a189-852915f4c727 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.208344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec28f4-56bd-48c7-a2c7-887ecacd1a4f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.254436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.255880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.256163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.256442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.259259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.260914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e363351-e2db-4b1f-b027-93315c7edccc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.262775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 196/397] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:19.264047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.264429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.264717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.283918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:19.285097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.335160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18008cc-6b72-4c49-a189-852915f4c727 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.337267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.368148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 197/398] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:19.369787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.377702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.382865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.382865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.382865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.382865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] RBAC: Adding query filter params (name=member) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:19.382865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.384571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.396335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 202/399] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:19.401521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.458461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.506415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7391af4-900e-4cb7-9c7c-7ea3a38947cb None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.541985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf741d-f70f-46c7-8ea5-5fa85f225c95 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.547441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.548468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.548853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.549229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/groups/42376ceca8464b31bceb2d35b76df506/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.551720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=92a7eab95c834cc293cbf8f4535c54e2, group_id=42376ceca8464b31bceb2d35b76df506, role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.553382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add3484-a724-4d23-9ee7-d75263d49af0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.579802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.580651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.580930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.581232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.582400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.583864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.585482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 203/400] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:19.586224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.608810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 198/401] 10.222.0.59 () {58 vars in 1442 bytes} [Thu Jul 14 10:58:19 2022] PUT /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2/groups/42376ceca8464b31bceb2d35b76df506/roles/d5708cbdc7894157a71389bd8517df5f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:19.612559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcefb-d62b-4b75-874f-e781f1637443 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.613557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcefb-d62b-4b75-874f-e781f1637443 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.614546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcefb-d62b-4b75-874f-e781f1637443 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.617276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 199/402] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:19.619152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bcefb-d62b-4b75-874f-e781f1637443 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.621688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.708306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007a64f-fe3c-4c3d-9e7e-d5047a863b9f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.745986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.746958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.747373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.747748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.756091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bcefb-d62b-4b75-874f-e781f1637443 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.760146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-45a9a882-b39a-43dc-af9b-5b113e7a9aaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:19.769663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:19.770654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:19.771493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:19.776609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 204/403] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:19.778448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.778743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.779017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.790911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.798798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:19.799486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.805833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.807095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.807746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.808227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.809798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.812509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.815285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 200/404] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:19.816515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.898669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 205/405] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:19.900572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.946123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.954680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f99e54-24f1-4a0a-b6d9-27eac121b8ed None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.002358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.003973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.004505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.005400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.006744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:20.007567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.009475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.027493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 201/406] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:20.027493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.037612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9bba2-7acc-4e93-a035-959b4346cf6c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.084746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.086176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.086870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.087334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.088923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/groups/42376ceca8464b31bceb2d35b76df506/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.091667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, group_id=42376ceca8464b31bceb2d35b76df506, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.094376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.135804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cd5a3-af71-440d-a7f1-2a88d90d9a10 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.172515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.173324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.173664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.173947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.190229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:20.197921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:20.198837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 202/407] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:20.204199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.216393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 206/408] 10.222.0.59 () {58 vars in 1442 bytes} [Thu Jul 14 10:58:19 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/groups/42376ceca8464b31bceb2d35b76df506/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:20.219511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.361864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec716b-abca-49a9-a88d-b82534bb0a10 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.379424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.403333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.405516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.406019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.406486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.407966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.409633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.412068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 203/409] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:20.417764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.425080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.447342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583dffc-2bce-46a3-9438-47070592a3a7 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.488283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.489331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.489855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.490328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.491317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:20.491837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.493404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.504900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 207/410] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:20.510510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.548310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800953-9b43-4cee-8f07-69d89302202f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.596282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.597169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.597433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.597714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.598919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.600424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.603108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 204/411] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:20.611381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.634201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f77a0-5523-4802-b7db-45a60154e01d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.677065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.699290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.699922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.700317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.700661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] PATH_INFO: `/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.702088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] RBAC: Authorizing `identity:get_group(group_id=3fb1dc7f3bc4468d9d794f04379fac9c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.703632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.705340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 208/412] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:20.712340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.739533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd9ed2-16a3-41b1-8476-68fb8a296aaa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.745096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.777980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.779469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.780031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.780676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/groups/42376ceca8464b31bceb2d35b76df506/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.783226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d, group_id=42376ceca8464b31bceb2d35b76df506, role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.784885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0ac8b-240c-456c-8d71-7bef8a476e41 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.868264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 205/413] 10.222.0.59 () {58 vars in 1442 bytes} [Thu Jul 14 10:58:20 2022] PUT /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d/groups/42376ceca8464b31bceb2d35b76df506/roles/d5708cbdc7894157a71389bd8517df5f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:20.869400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be100626-32ad-4e9c-b393-4ac4c96e4407 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.869796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976cee-90aa-4350-85ed-60c5bbaea470 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.870136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976cee-90aa-4350-85ed-60c5bbaea470 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.870419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976cee-90aa-4350-85ed-60c5bbaea470 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.871873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 206/414] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:20.878897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.879244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.880916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.899934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:20.901119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.917561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.918728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.919016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.919328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] PATH_INFO: `/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.921357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] RBAC: Authorizing `identity:get_project(project_id=7da05e4cf2b444729a9cbe3b6d5db535)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.922879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.925911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 209/415] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535 => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:20.933690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:21.037491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 207/416] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:21.060775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:21.083365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:21.103405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac399e1-0035-47c0-8e0a-7ff21deb4224 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:21.106780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:21.144933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:21.145838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:21.146120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:21.146396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] PATH_INFO: `/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535/groups/3fb1dc7f3bc4468d9d794f04379fac9c/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:21.150265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7da05e4cf2b444729a9cbe3b6d5db535, group_id=3fb1dc7f3bc4468d9d794f04379fac9c, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:21.151694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:21.169840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db2925-aa7f-4aa6-b7a9-8ac7effbb3b5 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:21.209776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:21.210586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:21.210924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:21.211229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:21.223072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:21.231676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:21.232564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:21.233905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 208/417] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:21.237719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 210/418] 10.222.0.59 () {58 vars in 1442 bytes} [Thu Jul 14 10:58:20 2022] PUT /identity/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535/groups/3fb1dc7f3bc4468d9d794f04379fac9c/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:21.240825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:21.353388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:21.359932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a569a-1ec7-4009-a024-8c8c02fdcbee None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:21.398789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:21.399575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:21.399783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:21.400065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:21.400880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:21.401198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:21.402595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:21.412648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 209/419] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:21.418550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:21.541388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8ddb-1734-4237-a74a-1fef8ef92db9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:21.578252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:21.578991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:21.579305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:21.579578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:21.591415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:21.655352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:21.656727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:21.798762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 211/420] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:21.804148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:21.921583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2ec71-535d-4749-b601-27ce233b6dc8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:21.957088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:21.957961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:21.958245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:21.958521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:21.959739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:21.961180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:21.963222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 210/421] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:21 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:21.968704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.036960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:22.105117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04555-3efb-45f0-b39d-575d414982fb None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:22.140996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:22.141859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.142137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.142407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.143603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:22.145053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf637-344d-44bf-87de-f948b87fc651 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:22.146970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 212/422] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:21 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:22.153256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.210152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:22.221094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be700-cebd-4c6b-a978-5f50b5188235 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.221380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be700-cebd-4c6b-a978-5f50b5188235 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.221795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be700-cebd-4c6b-a978-5f50b5188235 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.223209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 213/423] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:22.228421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.228709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.228988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.259203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:22.260425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.288069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f6d2d-8df7-4210-9add-08e87165ede5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:22.328218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:22.329244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.329727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.330227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.331300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:22.331987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:22.333836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:22.341093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 211/424] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:22.442000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 214/425] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:22.481177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.593961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:22.601054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362fd9-8b4e-4f60-be4b-5f147285a7f3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:22.637867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:22.639693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.640450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.640978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.652414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-a5b1b13d-23ab-42b5-bea4-d3bf938f273a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:22.661861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:22.662755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:22.663588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:22.665935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 212/426] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:22.671982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.714895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:22.749765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9d3f6-c894-4fe8-ab96-59c00b969ac5 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.750335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9d3f6-c894-4fe8-ab96-59c00b969ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.750812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9d3f6-c894-4fe8-ab96-59c00b969ac5 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.753100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 213/427] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:22.760980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.761540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.761829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.783189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:22.784746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.789953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db707-2e60-419c-95b8-21360d1dd6ac None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:22.846185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:22.846899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.847214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.847495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.848245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:22.848555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:22.849929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:22.862632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 215/428] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:22.867634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.900548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 214/429] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:22.925381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.993314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:22.997603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda0ba0-94ac-4e50-933b-81e5c08cc201 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.032772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.053369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.053567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.053778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.054037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.059179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec5776-11bf-49b9-a63f-98a93b21ed0c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.105617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.107084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.107597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.108628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.123282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:23.134442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:23.135372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:58:23.136153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:23.137844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 215/430] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:23.142530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.143152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.144668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.272411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.276288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347ae1e-ecaf-4911-8fbb-5576793bcb70 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.281985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 216/431] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:23.287144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.325405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.326332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.327147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.327614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.329389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] RBAC: Adding query filter params (name=member) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:23.329857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.331426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.343103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 216/432] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:23.348879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.360978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.419599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647cfc5-8466-4f56-bf03-d621233450c4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.466710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a3f80-3767-4b29-b1d7-d912f3769734 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.467981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.469314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.469767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.471147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] PATH_INFO: `/v3/projects/92a7eab95c834cc293cbf8f4535c54e2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.472538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] RBAC: Authorizing `identity:get_project(project_id=92a7eab95c834cc293cbf8f4535c54e2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.474150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.476927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 217/433] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/projects/92a7eab95c834cc293cbf8f4535c54e2 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:23.486168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.511705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.512605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.513055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.513399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.575506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.596258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.597751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f636ac3-9130-4568-990e-7fd53ad78904 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.605754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.612650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abaab4-905d-4b92-a622-f70af7662d59 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.651627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.653088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.653570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.654015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.656393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.658522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9834c1-516e-4581-b1bb-1ec05ded7da8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.660639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 218/434] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:23.661953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9e3c-b8ec-405c-a5ca-15b1298bdd7d None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.662392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9e3c-b8ec-405c-a5ca-15b1298bdd7d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.662802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9e3c-b8ec-405c-a5ca-15b1298bdd7d None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.667933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 219/435] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:23.668319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9e3c-b8ec-405c-a5ca-15b1298bdd7d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.709813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 217/436] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:23.709813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.709813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.709813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.726340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:23.727563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:23.786514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9e3c-b8ec-405c-a5ca-15b1298bdd7d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.803176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 218/437] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:23.803176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.836345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.836924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.837211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.837487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.838317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=92a7eab95c834cc293cbf8f4535c54e2) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:23.838627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.840084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213b091-8bb7-483e-b9e8-b47b03705762 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.843424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.845529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.846692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 220/438] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=92a7eab95c834cc293cbf8f4535c54e2&role.id=d5708cbdc7894157a71389bd8517df5f => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:23.847928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2294d-d040-4405-952f-d7729d3fa1a1 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.848214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2294d-d040-4405-952f-d7729d3fa1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.848489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2294d-d040-4405-952f-d7729d3fa1a1 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.851167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 221/439] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:23.851167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2294d-d040-4405-952f-d7729d3fa1a1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.922877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221579a-3f10-4961-a6b3-ddc6ca437584 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.961183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.962063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.962349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.962626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.963902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.965380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f72551-42bb-49ee-8123-93887f2f8cb7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.967256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 219/440] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:23.968727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.969007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.969284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.987201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2294d-d040-4405-952f-d7729d3fa1a1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.988839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:23.990272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:24.028701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.029796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.030413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.030413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.041884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:24.050825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:24.052333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 222/441] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:24.053100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.081539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 220/442] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:24.083469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.125449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.126244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.179017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb0f2-4c18-424e-b461-46192b854d06 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.209854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649306-9105-4665-b117-13106f4cd0ba None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.224256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.225125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.225409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.225687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.227484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.230719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.233563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 223/443] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:24.234355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.245883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.246565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.246854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.247197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.247983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:24.249042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.250451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.258939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 221/444] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:24.259816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.339817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1b2d-9e3f-427f-978a-9c1547ab36a8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.359607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.365549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7298d-3f39-440c-97fc-7a6277c5c629 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.367095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.375111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.375982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.376218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.376493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.387566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:24.395249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:24.396249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:24.397017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 224/445] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:24.397684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.401295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.401613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.401899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.402179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.403307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=f64ed86228d24c5b8f70e7a0e8625010, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:24.403630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.405693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.411823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 222/446] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=f64ed86228d24c5b8f70e7a0e8625010 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:24.412465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.507696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf161f-c7b6-4b62-87bc-563080446a28 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.544795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a3bc-0a58-415a-8b75-34e292625a94 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.546935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.547639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.547929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.548209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.588939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.589631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.589904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.590181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.590937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:24.591282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.593211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.601747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 223/447] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:24.607347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.616266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.632370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.632370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.633791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.721638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d58c4-615b-4c9a-999d-d9d60041e985 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.753169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 225/448] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:24.758799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.762178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.763237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.763237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.763459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.837779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.839297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.849921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.862130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c38757-8723-4a94-a2e5-3269c03c870c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.862581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.896973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.897748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.898028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.898298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] PATH_INFO: `/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.899497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] RBAC: Authorizing `identity:get_project(project_id=b1283d78ec2a4e52b5fe2b4e85318e8d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.900930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.902948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 226/449] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/projects/b1283d78ec2a4e52b5fe2b4e85318e8d => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:24.908614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.940998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 224/450] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:24.946352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:25.007752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aed4f-d21e-4d3e-8523-68650e73465b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:25.042340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:25.043220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:25.043501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:25.043774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] PATH_INFO: `/v3/groups/42376ceca8464b31bceb2d35b76df506` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:25.044901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c245-f717-4229-a9a3-4a320ab4fe7f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:25.045037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] RBAC: Authorizing `identity:get_group(group_id=42376ceca8464b31bceb2d35b76df506)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:25.046775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:25.048422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 227/451] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/groups/42376ceca8464b31bceb2d35b76df506 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:25.054001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:25.067499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:25.079512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:25.080745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:25.080847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:25.080847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:25.080985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] PATH_INFO: `/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:25.082161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] RBAC: Authorizing `identity:get_project(project_id=7da05e4cf2b444729a9cbe3b6d5db535)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:25.083690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:25.085430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 225/452] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/projects/7da05e4cf2b444729a9cbe3b6d5db535 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:25.090701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:25.154539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b39d62-8f38-4c28-b21a-4c35c1bb5864 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:25.192120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:25.192777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:25.193064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:25.193339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:25.193785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa1c06-a162-4356-a7e2-af78d4902f9c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:25.194161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] RBAC: Adding query filter params (group.id=42376ceca8464b31bceb2d35b76df506, role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:25.194469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:25.195940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:25.202483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 228/453] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:25 2022] GET /identity/v3/role_assignments?group.id=42376ceca8464b31bceb2d35b76df506&scope.project.id=b1283d78ec2a4e52b5fe2b4e85318e8d&role.id=d5708cbdc7894157a71389bd8517df5f => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:25.230914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:25.231825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:25.232114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:25.232500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] PATH_INFO: `/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:25.233684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] RBAC: Authorizing `identity:get_group(group_id=3fb1dc7f3bc4468d9d794f04379fac9c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:25.235199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfc618-16e2-452d-a7ff-0a7289a1264b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:25.237133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 226/454] 10.222.0.59 () {56 vars in 1180 bytes} [Thu Jul 14 10:58:25 2022] GET /identity/v3/groups/3fb1dc7f3bc4468d9d794f04379fac9c => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:25.242913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:25.335900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:25.347890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada643-6982-44a1-80aa-392a68cde81b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:25.383960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:25.384650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:25.384939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:25.385223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:25.386066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] RBAC: Adding query filter params (group.id=3fb1dc7f3bc4468d9d794f04379fac9c, role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=7da05e4cf2b444729a9cbe3b6d5db535) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:25.386380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:25.387892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a66dd-a9ca-49fb-a424-2348c852ae7d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:25.394649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 229/455] 10.222.0.59 () {56 vars in 1376 bytes} [Thu Jul 14 10:58:25 2022] GET /identity/v3/role_assignments?group.id=3fb1dc7f3bc4468d9d794f04379fac9c&scope.project.id=7da05e4cf2b444729a9cbe3b6d5db535&role.id=beb4a709a14b415db991ad1980b406a9 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:27.821099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4c96-848a-401c-900f-9fe9c304ea27 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:27.821614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4c96-848a-401c-900f-9fe9c304ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:27.821614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4c96-848a-401c-900f-9fe9c304ea27 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:27.823278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 227/456] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:27.836863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4e05-e77e-4082-851c-a85a735a8baf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:27.837371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4e05-e77e-4082-851c-a85a735a8baf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:27.837446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4e05-e77e-4082-851c-a85a735a8baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:27.860204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f35e4e05-e77e-4082-851c-a85a735a8baf None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:27.861404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f35e4e05-e77e-4082-851c-a85a735a8baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:27.932606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 230/457] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:27.969376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4c96-848a-401c-900f-9fe9c304ea27 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.044269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:28.088316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4c96-848a-401c-900f-9fe9c304ea27 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:28.123596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:28.124402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.124689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.124967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.134542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6319b2-b505-4c80-9520-6a6d1b4035ad None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.134921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6319b2-b505-4c80-9520-6a6d1b4035ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.135298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6319b2-b505-4c80-9520-6a6d1b4035ad None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.136971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 231/458] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:28.137334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:28.139698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:28.142741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.143034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.143356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:28.159027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:28.159960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:28.160792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:28.160792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:28.160792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:28.160792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:28.162832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 228/459] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:28.168129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.180216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:28.181422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:28.302801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 232/460] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:28.327213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.348634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:28.373877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63cad-54d9-4ce2-bb25-d5829a97fae9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:28.399027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:28.439059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:28.439755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.440042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.440327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.441054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:28.443595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:28.443595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:28.461088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 229/461] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:28.469619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.528619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3083db-6667-4c46-ab0e-e175f63a0b3d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:28.570303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:28.571717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.572757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.573213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.589569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:28.592631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:28.611452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867524-5b62-432d-8e2c-13922a6aa945 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-52620dc7-da1a-4992-a4fb-edf21d9b82a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:28.616964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:28.617780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:28.618648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:28.618648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:28.618648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:28.618648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:28.621561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 233/462] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:28.623800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09166-17da-4359-a64d-d25df2d270fa None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.624217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09166-17da-4359-a64d-d25df2d270fa None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.624615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09166-17da-4359-a64d-d25df2d270fa None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.626526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 234/463] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:28.627988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc0328-76be-44dd-bf1e-9e978f71a220 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.628415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc0328-76be-44dd-bf1e-9e978f71a220 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.628904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc0328-76be-44dd-bf1e-9e978f71a220 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.630443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 235/464] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:28.631607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc0328-76be-44dd-bf1e-9e978f71a220 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.656323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:28.657168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.657461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.658069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.659069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:28.661009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b82ca-63a7-40fa-86e8-02c03f9bd887 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:28.699585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:28.737582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc0328-76be-44dd-bf1e-9e978f71a220 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:28.771610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:28.771946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 230/465] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:28 2022] POST /identity/v3/users => generated 246 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:28.772779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.773255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.773378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.773572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.773858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.774046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.774981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:28.775471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:28.777058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213444a-ebec-48da-9433-e48c120da803 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:28.795391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 236/466] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:28.795391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:28.795391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:28.795391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:28.811089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:28.812545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:28.815709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:28.817110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:28.842337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:28.906623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 237/467] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:28.907726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.916517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 231/468] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:28.937737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:28.979494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.043511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518acc10-2a7e-489f-801e-bb3e63c3d25d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.065811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63774978-4415-47e9-84b8-4c9b64ee8714 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.087596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.088556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.088854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.089139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.089861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.091849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.110124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.117195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.117195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.117195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.122713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.124913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:29.144071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:29.144935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:29.146038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:29.146038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:29.146038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:29.146038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.146038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 232/469] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:29.146639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.158831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.169510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 238/470] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:28 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:29.176494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.319825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f0d7-d566-46d3-acb3-d9cae49b4910 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.365765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2e6f9-1e7e-4cea-916e-afcb785e028d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.369097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.369897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.370181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.370457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.379405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.384366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.385927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-256396f1-d79a-4387-9786-670d23bb8dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:29.406547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:29.407501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:29.408522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:29.408522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:29.408522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:29.408522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.408522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 233/471] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:29.409386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5720cb-63f5-4f28-afd1-9eb49e3f524c None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.409682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5720cb-63f5-4f28-afd1-9eb49e3f524c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.409882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5720cb-63f5-4f28-afd1-9eb49e3f524c None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.414397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.417084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.417084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.417084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.417304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 234/472] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:29.417391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:29.417391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.417512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5720cb-63f5-4f28-afd1-9eb49e3f524c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.418342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c85448-27ed-416b-98c1-623b0c986492 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.437760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 239/473] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:29.440475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.440779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.441065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.459020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:29.460256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:29.509820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.534506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 240/474] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:29.539691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.566568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5720cb-63f5-4f28-afd1-9eb49e3f524c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.613508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.614870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.615407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.615879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.617340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:29.617855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.620009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.639515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 235/475] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:29.640892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.669316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.672882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0902b-cb6d-4bff-acae-244d7cc36978 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.713099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.713883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.714172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.714449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.715390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.716767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012ed2-3673-46a8-8d7f-d879d35baee5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.794172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:29.799857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 241/476] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:29 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:29.800900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.823938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.825024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.825024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.825288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.826294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.827956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:29.874307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.sql.core [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:58:29.874307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:58:29.874307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [parameters: {'id': '0bd8e263c5e64ee4a89c69fbfabd3dd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:58:29.874307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80483) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result.read() Jul 14 10:58:29.906442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:58:29.907509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application yield resource Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise self.value Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application session.commit() Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:58:29.908436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:58:29.909345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:58:29.910201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result.read() Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:58:29.911010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [parameters: {'id': '0bd8e263c5e64ee4a89c69fbfabd3dd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:29.913242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:58:29.915069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:29.921290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 236/477] 10.222.0.59 () {60 vars in 1131 bytes} [Thu Jul 14 10:58:29 2022] POST /identity/v3/roles => generated 144 bytes in 267 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:29.921290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:29.948880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c980a44-3d52-4157-a459-330e83bee53a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:29.992263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:29.993660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:29.994572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:29.995159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:29.996384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:29.999029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:30.084223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 242/478] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:29 2022] POST /identity/v3/users => generated 249 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:30.089503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c60e-9619-40c1-a66d-5abf33d7924d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:30.130610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:30.131480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:30.131765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:30.132042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:30.144710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fd27be70-d663-476f-abf3-8d7e8be3122a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:30.153335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:30.154877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:30.155880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 237/479] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:58:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:30.161540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:30.191145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:30.301136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb305-ecd0-4cba-a946-5895fb1c8898 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:30.337883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:30.338563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:30.338850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:30.341192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:30.341192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:30.341192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:30.342459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7768-7d5a-492f-9248-bca4ea4c45e5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:30.352024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 243/480] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:31.052941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb3f95-a122-4bac-85f2-1e4d2c074999 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.053466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb3f95-a122-4bac-85f2-1e4d2c074999 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.053549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb3f95-a122-4bac-85f2-1e4d2c074999 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.055148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 238/481] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:31.061224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.061667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.061851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.079640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:31.080859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:31.152909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 244/482] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:31.177107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb3f95-a122-4bac-85f2-1e4d2c074999 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:31.226003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:31.306134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb3f95-a122-4bac-85f2-1e4d2c074999 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:31.355838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:31.356648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.357023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.357804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.377977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:31.385702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:31.386584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:31.387376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 239/483] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:31.393462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:31.405023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:31.509075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d544b29-5a93-491d-a15d-5614e6fa16b6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:31.561313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:31.561927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.562216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.562496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.563278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:31.563596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:31.564975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e94f26-cd36-433b-8786-f1014ae270c4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:31.573809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 245/484] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:31.580258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:31.581350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717079c-e4e0-4efc-9976-1785b0a5d496 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.581645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717079c-e4e0-4efc-9976-1785b0a5d496 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.581926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717079c-e4e0-4efc-9976-1785b0a5d496 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.583298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 246/485] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:31.592233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.592525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.592802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.620212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:31.621409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:31.699499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:31.711711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 247/486] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:31.751994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be9e6b-03f1-4695-95eb-61dbe7b102fb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:31.754555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:31.762338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:31.819951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:31.820682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.820968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.821247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.840429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:31.842593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:31.870974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:31.871907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:31.872775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:31.872775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:31.872775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:31.872775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:31.876693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 240/487] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:31.882327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:31.887576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b84751-e864-410e-939d-46945f61afd8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:31.948323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:31.949564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:31.950211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:31.950556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:31.976961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:31.984841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:31.985806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:31.986599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:31.993012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 248/488] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:31.993985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.041722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844d777-325d-471d-b623-226999f2bf07 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.089519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.092893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.094045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.094476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.094895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.095847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:32.103141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.104764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.123625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 241/489] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:32.130759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.151537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.157154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0185c-c894-44b9-8285-019bb0eb06b0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.236236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.236875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.238497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.239889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:32.240706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.242810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.261829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 249/490] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:32.267394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.318954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd8c24-d2c7-4236-99cd-2a73186ac21b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.392486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9db-c525-4868-b825-9d0c1a210450 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.393946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.394784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.395048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.395358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.412290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.413770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.414571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.433446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.434575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.435160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.436217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.447785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.449585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:32.452475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:32.453379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:32.454317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 242/491] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:32.459543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:32.473299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:32.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:32.475105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:32.475105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:32.475105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:32.475105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:32.477371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 250/492] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:32.483640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.599439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7fe41-bc1e-48c7-831c-02998e046e6b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.636224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.637427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.637908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.638432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.639384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:32.640278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.641862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.656595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 251/493] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:32.663279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.698007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.736730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.779167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22170b27-7ebb-4bb0-8f74-308f9ab973e0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.811089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb3925-8bd4-4b0e-8809-22f6adc55a35 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:32.846948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.848034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.848395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:32.848533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.848892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] PATH_INFO: `/v3/users/glance` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.849084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.849374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.849656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.850510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:32.850828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.852322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7dac3-62da-4fbd-90e8-a8b3327a485c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.857104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:32.858705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-331507d0-b131-4ff8-b65a-b24d22897aec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:32.876909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:32.877857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:32.878954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 252/494] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:32.879823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5919-39bb-4872-800b-4342a0bb66b5 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.880107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5919-39bb-4872-800b-4342a0bb66b5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.880382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d5919-39bb-4872-800b-4342a0bb66b5 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.881806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 253/495] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:32.887163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9a135-4705-45f0-9205-9cb764efb1c7 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.887163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9a135-4705-45f0-9205-9cb764efb1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.887163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9a135-4705-45f0-9205-9cb764efb1c7 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.887163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 254/496] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:32.887163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9a135-4705-45f0-9205-9cb764efb1c7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:32.904133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 243/497] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:32.909952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5426f7-c465-4845-8c87-bb955ee0a2fc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:32.910263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5426f7-c465-4845-8c87-bb955ee0a2fc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:32.910602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5426f7-c465-4845-8c87-bb955ee0a2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:32.927825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2d5426f7-c465-4845-8c87-bb955ee0a2fc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:32.929141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2d5426f7-c465-4845-8c87-bb955ee0a2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:32.982793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:32.993663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9a135-4705-45f0-9205-9cb764efb1c7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.002642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 244/498] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:33.005252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bae18-e54b-4e1d-a974-1c2e109642e5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.005252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bae18-e54b-4e1d-a974-1c2e109642e5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.005252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bae18-e54b-4e1d-a974-1c2e109642e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.020929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-015bae18-e54b-4e1d-a974-1c2e109642e5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:33.022221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-015bae18-e54b-4e1d-a974-1c2e109642e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:33.031432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.032107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.032390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.032665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.033506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:33.033818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.035279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.071956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 255/499] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:33.072777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.085578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.093451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 245/500] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:33.094742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bec36-9ba3-43ee-8e51-f6d8ffd322a8 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.100282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bec36-9ba3-43ee-8e51-f6d8ffd322a8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.100282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bec36-9ba3-43ee-8e51-f6d8ffd322a8 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.100282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 246/501] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:33.100282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bec36-9ba3-43ee-8e51-f6d8ffd322a8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.180860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3908a6-bd6e-4a09-be7c-37e8e85bd967 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.206891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bec36-9ba3-43ee-8e51-f6d8ffd322a8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.216993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.217641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.217924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.218199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.235360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.241336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.242180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.242544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.242731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.253041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:33.260719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:33.261912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:33.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 247/502] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:33.264281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.291716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.293183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369beb7-0fd1-45d3-9333-b97b4c4cae11 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.321190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.366574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbee3cb-b25a-496c-ab7f-ce4e899b5543 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.392003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 256/503] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:33.393381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.393672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.393954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.402375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.403023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.403337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.403640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.410351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:33.411594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:33.455310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.471767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.473243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.477398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 257/504] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:33.478207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.528811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.573488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 248/505] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:33.574318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.581745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74bb44-96b3-4633-a3e0-5286c76405ca None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.614958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.615788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.616077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.616354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.626968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:33.634724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:33.635643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:33.636443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 258/506] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:33.637113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.680442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.680907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd02c6d-95e9-49eb-b479-450c405e58ea None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.714590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.715211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.715505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.715777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.716555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:33.716857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.718274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.726110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 249/507] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:33.726890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.743683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38202-8728-405d-b71d-a21809fda0d8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.746415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.778120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.779175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.779460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.779730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.789451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.790910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:33.809000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:33.810003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:33.811386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:33.811386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:33.811386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.811386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:33.811386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 259/508] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:33.812133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.834551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d12aa2-d8c4-4247-af5e-9d001bd59a2f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.867957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.868747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.869047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.869328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.879981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.881474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:33.899580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:33.900609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 250/509] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:33.901607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.904619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.915474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef48ad-a5e0-44ad-adec-acc1df4ec81c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.949363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.950323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.951282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.951566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.961591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.963051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.969047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:33.980975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:33.982011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:33.982842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:33.982842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:33.982842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:33.982842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:33.982842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 260/510] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:33.983094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.007817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f8860-9754-45e3-ba81-30cbcd953cf7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.041342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.042007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.042292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.042571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.043351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.043674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.045269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.053104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 251/511] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.053888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.083762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed752d-11f6-4f9e-b53b-4a1e4f9d3079 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.116650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.117424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.117705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.117978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.129752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.131218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.132928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:34.148617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:34.149527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:34.150395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:34.150395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:34.150395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:34.150395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:34.150395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 261/512] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:34.150678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.158735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb280b-3797-4e36-b2c8-927dda6ce4e7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.193064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.198412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.199078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.199398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.199673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.200395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.201395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.203020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.214306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 252/513] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.215091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.252713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a2d0-d747-457b-97ec-548831e5b57b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.286377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.287030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.287342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.287619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.288334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.288959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.289996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.302251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 262/514] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.303035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.313513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b261d1-646e-4fec-aa3d-7564f2520717 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.348791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.349455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.349743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.350029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.350924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.351275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.352788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.356277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.365166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 253/515] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.365970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.460312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a431ba-7243-4fb9-b599-9a72f3906bac None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.481586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.499162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.499959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.500243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.509477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.511046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.512608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:34.531209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:34.532052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:34.532896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:34.532896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:34.532896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:34.532896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:34.533110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc14a09-6a24-444f-b6eb-86bfffc06907 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.533179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 263/516] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:34.533179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.565977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d99647-b618-41ad-9083-87409b849ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.566635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99647-b618-41ad-9083-87409b849ade None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.566920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99647-b618-41ad-9083-87409b849ade None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.567226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d99647-b618-41ad-9083-87409b849ade None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.567941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d99647-b618-41ad-9083-87409b849ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.568247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d99647-b618-41ad-9083-87409b849ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.569611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d99647-b618-41ad-9083-87409b849ade None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.581904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 254/517] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.582659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d99647-b618-41ad-9083-87409b849ade None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.636981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6ae9-6d5d-419d-a228-d2a4576b54a0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.644531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.671652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.672441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.672730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.673006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.679586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.681022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.681675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d99647-b618-41ad-9083-87409b849ade None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.693202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:34.695678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:34.696513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:34.697349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 264/518] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:34.698746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.718121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.718811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.719367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.719869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.720842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.722292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.787918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 255/519] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/users => generated 247 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:34.788661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.803281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdcf52-eaf7-410d-8319-93fda203500e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.845809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.846994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.847462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.847883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.856230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.858240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:34.875412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:34.876514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:34.877544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:34.884054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 265/520] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:34.885244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.918488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.921846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f408055-764e-470a-ad06-8e8390627446 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.964230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.965179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.965179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.965445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.966710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.966816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.968808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.980938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 256/521] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.982055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.011049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f1873-87d0-404e-b659-fde8b24e1c80 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.015550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.049695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.050425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.050708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.050985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.058653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.060716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:35.076961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:35.077973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:35.078864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 266/522] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:35.080067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.100700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ae0de-8d34-42a9-a52b-04b0ffe2c417 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.138802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.139506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.139799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.140077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.140882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.141187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.142657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.161642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 257/523] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.162479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.191611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.207761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af6466-c436-42f8-a149-eba6b18cfcac None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.245397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.246064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.246348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.246624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.248468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.248798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.250247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.261947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.271275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 267/524] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.272143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.282280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda74bb-bd98-479b-9800-9afed37f7529 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.320664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.321740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.322034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.322315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.330687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.332188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:35.347238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:35.348156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:35.349081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 258/525] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:35.349712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.386902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a8a50-6754-4438-a8dd-1ff65f1dc043 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.424406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.425077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.425356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.425625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.426456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.426761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.428217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.451151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 268/526] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.451151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.455779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.460371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbee8a2-ac97-4ed3-9b12-d415dafeeaed None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.496096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.496761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.497051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.497998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.498844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.499188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.500638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.511154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 259/527] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.511154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.531044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.595831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f0f03-582e-46f3-8a0a-4b3803e3ed3f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.633809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.634683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.635402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.635691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.636846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=bbfd6e827b9e49949cd6543edc89bbaf) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.637168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.639784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.645839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10038bc1-7d82-4197-8baa-1b00b9cbf755 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.647353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 269/528] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/role_assignments?user.id=bbfd6e827b9e49949cd6543edc89bbaf&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.648526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.699287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.702052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.702341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.702625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.703538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:35.703864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.705376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.734064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 260/529] 10.222.0.59 () {56 vars in 1139 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:35.740237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.755193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.798432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6aec-9915-423d-a7ba-cd152ac7fcff None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.848069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:35.851620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.852282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.852571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.852846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.924128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a564cd-12db-4626-ae1c-b6bf26c6310e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.940082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.941891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.997958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.999940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:36.000229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:36.000515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:36.040087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.080879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 270/530] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:36.093043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.147008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:36.148443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:36.267729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.306904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 261/531] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:58:36.309261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759757a5-7f64-4889-ade7-bf32529b09aa None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:36.314175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.383311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:36.384109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:36.384394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:36.384664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:36.412757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:36.414134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:36.451139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:36.452042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:36.454416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:36.454416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:36.454416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:36.454416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:36.454416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 271/532] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:36.474409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.495995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321c0-dca5-4bcd-867a-18c824fb3719 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:36.511830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.562641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:36.563481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:36.563769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:36.564047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:36.576374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:36.577949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:36.596726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:36.597586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:36.598402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:36.598402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:36.598402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:36.598402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:36.598402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 262/533] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:36.604808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.647526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.705421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53d57-be58-43ae-949c-838b86b62eb8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:36.725361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b4701-aa2f-471c-96f2-7d05e595ba36 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:36.750281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:36.750970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:36.751289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:36.751571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:36.752310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:36.752621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:36.754199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:36.766114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:36.767136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:36.767639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:36.768104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:36.769134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:36.769635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:36.771441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:36.774663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 272/534] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:36.795164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 263/535] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:36.795441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.796610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:36.915861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.945761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:36.951057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd2ea-eb84-4fc1-8fa3-24622cf2a22b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:36.977409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca848b-b0d1-4316-a409-1630212c3802 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.016854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.017660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.017943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.018218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.029588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.031100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:37.046889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:37.047895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:37.048743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 264/536] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:37.054714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.058264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.059062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.059995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.060292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.076572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.078046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:37.106315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:37.107692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:37.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 273/537] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:37.116667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.230341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f08f69-7aa3-41ff-ba72-66c5ad38727e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.256308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410fca2-8cbe-4bc2-b5bb-3b9f3250543a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.272482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.273181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.273470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.273749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.275022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:37.275368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.277357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12169366-4a7b-45de-a1ea-65cc2e80e6a8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.290423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:37.299198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 265/538] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:37.300780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ee76-b9aa-4284-9012-d2ae23ddd1dd None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.301225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ee76-b9aa-4284-9012-d2ae23ddd1dd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.301999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31ee76-b9aa-4284-9012-d2ae23ddd1dd None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.307252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 266/539] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:37.307369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.307369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.307369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.307369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.308100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31ee76-b9aa-4284-9012-d2ae23ddd1dd None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.308903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:37.309384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.312025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e315bd-02af-40e9-aa97-84f210e7058f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.325188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:37.335174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 274/540] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:37.336925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.342954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.342954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.357637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:37.360158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:37.420397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31ee76-b9aa-4284-9012-d2ae23ddd1dd None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.438992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 275/541] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:37.440550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.474060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.474832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.475202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.475545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.476496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=c86f8e914f3f4312a60791af443b2f5d) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:37.476872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.478517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.493404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 267/542] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/role_assignments?user.id=c86f8e914f3f4312a60791af443b2f5d&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:37.500463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.624986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa73865-4489-4a8b-9f2a-340ea105ca3a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.647533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:37.687291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:37.693451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.694385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.694890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.695388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.696399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=0a463c3c3f974bae83a6f4b21729f503) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:37.696891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:37.698635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:37.701365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d529f0-52a1-4b57-8acd-ee5bea602b09 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.707168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 276/543] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/role_assignments?user.id=0a463c3c3f974bae83a6f4b21729f503&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:37.748456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:37.749265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.749555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.749839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.769935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-5d35e9ba-9f87-48b2-8cdf-b2296df1b8b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:37.786399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:37.787429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:37.788235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 268/544] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:37.794239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:37.810468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1938-1485-4e64-b7b4-b1efee424c74 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.810778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1938-1485-4e64-b7b4-b1efee424c74 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.811035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e1938-1485-4e64-b7b4-b1efee424c74 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.816508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 269/545] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:37.818251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:37.818544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:37.818828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:37.847809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:37.849063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:37.955874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 270/546] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:37.973166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaab66-0655-499d-8a18-b6d5d21e1aef None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:37.998137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.018863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.019564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.019852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.020137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.020900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:38.021215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.022731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2003348-87d8-4742-b4eb-2ab8a65fb3df None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.032941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 277/547] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:38.034475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf54d4-5b8d-42bc-8b43-7c21eb4b2dca None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.034766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf54d4-5b8d-42bc-8b43-7c21eb4b2dca None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.035047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf54d4-5b8d-42bc-8b43-7c21eb4b2dca None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.036601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 278/548] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:38.040733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf54d4-5b8d-42bc-8b43-7c21eb4b2dca None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.072489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.103602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.121342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16457d-e3b8-4694-8800-7889b031fe9f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.147149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf54d4-5b8d-42bc-8b43-7c21eb4b2dca None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.160222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.161366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.161858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.162336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.174189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-4eee63b4-5b70-41c3-a207-87ddbd1045fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:38.183782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:38.185175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:38.186002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:38.188095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 271/549] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:38.190292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.190592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.190877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.192884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.194091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.194381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.194663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.211915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.212373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:38.213387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.214006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:38.236975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:38.237890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:38.238742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:38.238742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:38.238742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:38.238742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:38.242967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 279/550] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:38.244000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.295843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 272/551] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:38.296674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.357850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.381828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a64a8-319a-4a37-b24a-e32a0770dfd6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.392528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.419385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.420137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.420336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.421614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.422441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:38.422761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.424234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.432284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662efdb-4f59-41a5-9758-d4e9ba49e73a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.433041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 280/552] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:38.433914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.483926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.485116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.486096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.486818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.487968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:38.488452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.490348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.512184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 273/553] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:38.513792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.548311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7043874-06a8-4712-9412-e5d7cc7d6161 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.585045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.586083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.586203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.586390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.597275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:38.604877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:38.605955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:38.607092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:38.611074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 281/554] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:38.611981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.625560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72732-efed-4bb4-91ff-80806f953e08 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.628218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.647334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.661894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.662788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.662978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.663331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.674358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.675855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:38.694340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:38.695558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:38.696433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:38.696433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:38.696433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:38.696433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:38.696433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 274/555] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:38.696819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.727394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62705491-0f7b-4c7f-8849-22d2aafd135b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.766438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.767575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.768021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.768439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.780453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.783081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:38.800713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:38.801738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:38.803017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:38.805806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 282/556] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:38.806800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.814172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743afb82-69eb-4b09-9946-ac770c47e500 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.850616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.851329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.851620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.851916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.852661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:38.852976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.854355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.868949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 275/557] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:38.868949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.892411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.921009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.932428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac00e-1b3b-4866-bef1-7f1fb3c48525 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:38.975036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:38.976128 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.976663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.977141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.978082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:38.979034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:38.980776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:38.994165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 283/558] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:38.995417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.007957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2a898-adbe-46ad-a100-c0767ab6ab1b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.054929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.055479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.056014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.056295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.057144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:39.057815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.059984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.080237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 276/559] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:39.081646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.115998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe07c6-c0c2-4582-bec3-7e41c6c12aab None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.151679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.152736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.153321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.153739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.163653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.165702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.167404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:39.190354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:39.191496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:39.192424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:39.192424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:39.192424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.192424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:39.198665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8272-b5b2-4612-8fb1-d03a1cdba24b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.198882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 284/560] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:39.199930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.203097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.236174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.236927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.237330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.237626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.245265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.248245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:39.263980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:39.265051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:39.265986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:39.270012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 277/561] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:39.270728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.319661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae040a00-836a-47a2-96f3-fb158b52bfc9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.360371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.361485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.362045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.362520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.381679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4c823-2aa4-410d-a051-3622240e267f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.419869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.420479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.420838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.421092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.421800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:39.422192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.423501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.426516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.437521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 278/562] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:39.440932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.448788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.451523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.454181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.551622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754d90-f99e-470c-bb0c-6eeecbf2365a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.586401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 285/563] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:39.587414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.601300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.602530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.603521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.604277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.605586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:39.606157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.608790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.634324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 279/564] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:39.635530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.702430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.711477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417da94-278d-43e6-af2d-1c1f92cdb681 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.713135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.754919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.755652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.755938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.756214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] PATH_INFO: `/v3/users/glance` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.764713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.766346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.773498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b4e4-b8e9-418d-a186-e71b5d8ce125 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:39.781845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:39.782964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:39.784114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:39.787440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 286/565] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:39.788449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.814420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.815906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.815906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.815906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.829580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.832002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-80cd6e1e-0c46-4c78-89b5-77d9a4c77358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:39.857182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:39.858097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:39.858980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:39.858980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:39.858980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.858980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:39.858980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 280/566] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:39.860778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd80fb-69e9-440c-bb7e-b77744cf3387 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.861547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd80fb-69e9-440c-bb7e-b77744cf3387 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.861718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd80fb-69e9-440c-bb7e-b77744cf3387 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.863807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 281/567] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:39.864447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd80fb-69e9-440c-bb7e-b77744cf3387 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.912188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f4f42-a035-4491-8e86-50887a6063ea None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.947045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.947864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.948142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.948412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.961183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.962688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:39.981486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:39.982618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 287/568] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:39.984184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:39.996585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd80fb-69e9-440c-bb7e-b77744cf3387 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.001330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.004401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.033940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.034712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.035005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.035303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.036271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:40.036577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.038686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5479-b18a-4367-bdd4-76aabfcefb39 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.057713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 282/569] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:40.059778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa9be-f37c-4301-bcf6-b1da96c29a94 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.060076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa9be-f37c-4301-bcf6-b1da96c29a94 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.060356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaa9be-f37c-4301-bcf6-b1da96c29a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.078224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dceaa9be-f37c-4301-bcf6-b1da96c29a94 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:40.079681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dceaa9be-f37c-4301-bcf6-b1da96c29a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:40.100686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a83119-737d-42dd-be96-782bf35859e8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.139556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.140517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.140967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.141431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.142569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:40.142878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.144410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.158477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 283/570] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:40.158656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 288/571] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:40.159970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8fd2a-3e56-4610-bcaf-b57ede546b05 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.160362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8fd2a-3e56-4610-bcaf-b57ede546b05 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.160978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8fd2a-3e56-4610-bcaf-b57ede546b05 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.162528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 284/572] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:40.163202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8fd2a-3e56-4610-bcaf-b57ede546b05 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.168106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.250409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.276181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.276780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93ce61-3bd3-4472-a71d-92ed2e874215 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.284042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8fd2a-3e56-4610-bcaf-b57ede546b05 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.314793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.315765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.316233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.316690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.317933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:40.318459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.320873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.323365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.324214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.324499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.324792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.334869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 289/573] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:40.335849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.338419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.338419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-d7b00f39-bcac-4f13-ac02-6ef9c2cfca7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:40.361275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:40.365314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 285/574] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.366193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.384706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:40.386239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:40.445498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2635a-c152-4b0e-92d5-0ebd7bd96057 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.456340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 286/575] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:40.457188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.501722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.503275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.503561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.503838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.505834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.511599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.513824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:40.530237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:40.532013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 290/576] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:40.533236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.542418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.568918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e257f1-8940-4522-96f8-09d31a252fa1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.604853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.605653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.605939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.606725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.617265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:40.625506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:40.626635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:40.627549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 287/577] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:40.628258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.646494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af54270-7559-442d-b71d-c74584db0404 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.680978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.681807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.682103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.682385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.690480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.691983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:40.706024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:40.706950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:40.707896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 291/578] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:40.708394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.728304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c79230-59fc-4243-bd93-1775e398a788 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.762637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.763347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.763616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.763888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.764608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:40.764828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.764955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.766670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.768997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.779268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 288/579] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:40.780200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.819958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b40d0-083d-4f22-9bbd-5c49c5c4974d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.853796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.854682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.854977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.855289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.866415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:40.874362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:40.875321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:40.876164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 292/580] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:40.876845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.890360 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcce73-1496-4d72-b280-5de8a5f19a38 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.927002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.927697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.927980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.928257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.929072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:40.929385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.930819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.942403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 289/581] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:40.943225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.977147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33691bc-a417-4ae6-aef2-4719a87cc450 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.991247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.000133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.011470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.012141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.012476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.012756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.013506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.013816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.015205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.023432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 293/582] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.024059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.050282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a4af9-db62-4035-8516-ca2cb49b82bd None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.083003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.083692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.083982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.084259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.085058 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.085377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.086811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.099204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 290/583] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.099998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.127145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577172cc-128a-4103-91cd-7f60c9564b19 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.161045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.161827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.162107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.162387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.169599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.171030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:41.185262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:41.186149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:41.187036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 294/584] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:41.187544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.211026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1f479-72bb-46c4-904f-387eeb4bee3c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.222275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.228485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.245917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.246613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.246903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.247221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.248078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.248398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.249860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.257781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 291/585] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.258700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.287948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8cec-c8dc-4587-b357-4377e759edde None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.321020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.321663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.321941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.322213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.323034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=7e631e04eeba421885436f9f2a1330b3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.323372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.325555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.330990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 295/586] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/role_assignments?user.id=7e631e04eeba421885436f9f2a1330b3&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.331772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.370428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27cf6-b155-46ed-a6c2-5dc2b26fe41f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.412116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.412933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.413219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.413496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.425438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.426941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.443882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:41.445528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:41.446402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:41.447333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:41.447333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:41.447333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:41.447333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:41.447333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 292/587] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:41.448013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.450587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587107ba-a5af-4aa3-8f85-48665b4bd1ab None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.485938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.498754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.499670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.500432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.500432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.519547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=c101547390374884b75d95b637267e79, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.520991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.556841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878894cf-ffdb-4cc2-a8c8-f0737b08062c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.591256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.591922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.592244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.592524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.593313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.593628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.595073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.611151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 296/588] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:41 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:41.611151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.619362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 293/589] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.620334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.756243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746dca6-0715-4eda-88a3-2485f249a05e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.759774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.774464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51504-d216-43ae-b641-e12233cdbd50 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.807078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.807928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.808177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.808449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.822654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.822654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.826771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.827585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.827876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.828158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.828890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:41.829202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:41.831232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:41.841931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:41.842839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:41.843772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:41.843772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:41.843772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:41.843772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:41.843772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 297/590] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:41.844456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.845660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 294/591] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:41.847372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:41.848701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.955162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394eec9-dfe6-4451-b1d6-3e7a84af97ae None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.962276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bacd-b1da-4f1a-a339-8d0ec971d095 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:41.991989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.993292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.993791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.994347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/bbfd6e827b9e49949cd6543edc89bbaf/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.998109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:41.998812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.999782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.999782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.000101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:42.000412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.001815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.012280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=bbfd6e827b9e49949cd6543edc89bbaf, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.014189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.019885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 295/592] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:42.019885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.034183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.096246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 298/593] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:41 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/bbfd6e827b9e49949cd6543edc89bbaf/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:42.097355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.142308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b922b-1cfb-4005-8525-56232ed620ff None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.192246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.194687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.196065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.196584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.197061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.207618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.214149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:42.234654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:42.235709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:42.236647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:42.237283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 296/594] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:42.243672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.296053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f9a19-2e74-482b-98f3-32935a094024 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.339101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.339943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.340248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.341023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.348098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.351064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.352614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:42.372007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:42.373090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:42.374039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 299/595] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:42.379351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.384366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d07bb-1ee1-4bfc-9a22-1a73a1471557 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.451861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.452701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.452872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.453145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.453999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:42.454314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.455845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.496508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9169-408d-40ea-8347-20a44ddf94ae None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.503147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 297/596] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:42.516603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.517398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.540608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.542047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.542608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.543088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.544866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:42.545400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.547310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.567783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 300/597] 10.222.0.59 () {56 vars in 1139 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:42.574518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.697671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ae0f-276f-4fe2-a1f4-b63b22349d16 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.717270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9fbc7-cd2c-4ae6-bb2b-85f01e216761 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.735566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.736764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.737266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.737744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.749798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.752003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.769587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:42.783837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:42.784780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:42.785770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:42.785770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:42.785770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:42.785770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:42.786449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 301/598] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:42.790410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.799064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.803997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.804282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.804574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.824074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.825909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.828326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:42.859143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:42.860089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:42.860938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:42.860938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:42.860938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:42.860938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:42.868893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 298/599] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:42.880086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:42.933167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ee3a6-f71b-45a2-b930-abda882c8f95 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.974259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.975088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.975508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.975889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.977014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:42.977428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.979386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.993158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 302/600] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:43.001606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.083929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4048d20-c926-4653-b965-d1c95b536e5a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.118435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.119376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.119832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.120256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.121119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:43.122111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.123816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.136415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 299/601] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:43.144880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.217308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:43.230368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:43.252770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c201a-3696-4a00-95a0-f8d26e0dce0e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.302888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.303730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.304016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.304296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.311741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.313212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:43.328225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:43.329116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:43.329984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:43.330403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d12b8-a881-4caf-a711-8f4d33136e07 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.330475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 303/602] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:43.337021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.387551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.388370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.388655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.388934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.398831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.399132 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:43.417265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:43.418161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:43.419647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 300/603] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:43.424070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.496563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62707c-e374-4ee3-94f5-45e6fbe1aa0a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.582253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:43.585999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a25d8a-2e85-46a3-86d3-397c5f43c744 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.589722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.590387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.590662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.590944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.591781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:43.592098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.593552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.624307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.624892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 304/604] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:43.625003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.625257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.625543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.626433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:43.626775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.628310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.632000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.639443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:43.643464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 301/605] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:43.649313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:43.758162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf1d9-1a0c-4cc7-b48e-3b8199603c47 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.768079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c023fc0-61f1-4992-abf1-b4d85ca846a7 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:43.795204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.796158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.796450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.796740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/c86f8e914f3f4312a60791af443b2f5d/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.808169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:43.810034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.810499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.810592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/0a463c3c3f974bae83a6f4b21729f503/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.813079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=c86f8e914f3f4312a60791af443b2f5d, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.814604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165bd0f-2dc3-4a9a-b5d3-4a876cb4f3da None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.827021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=0a463c3c3f974bae83a6f4b21729f503, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:43.828641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076ea02-f987-4206-9231-9b4a0eeeb9cb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 302/606] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:43 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/c86f8e914f3f4312a60791af443b2f5d/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ec-0be5-4e7b-8a25-c29c48360445 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ec-0be5-4e7b-8a25-c29c48360445 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ec-0be5-4e7b-8a25-c29c48360445 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 303/607] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:43.915273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738868e9-d1df-4814-8cee-13c3fa340086 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.915754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738868e9-d1df-4814-8cee-13c3fa340086 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.915754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738868e9-d1df-4814-8cee-13c3fa340086 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.915754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 304/608] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:43.916633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.916924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.917202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.937161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 305/609] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:43 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/0a463c3c3f974bae83a6f4b21729f503/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:43.938487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:43.938778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:43.939055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:43.948072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:43.951207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:43.961970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:43.975310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:43.976555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:44.087860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.102489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 305/610] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:44.116147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 306/611] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:44.132249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.144561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.258488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9309b-2b5c-49af-87fc-f74793facf5c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.293673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d037540-b7c5-4429-ad11-b5573ac961df None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.331890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.332705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.332995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.333274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.347171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.351657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:44.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:44.365110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:44.365949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:44.367752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.368550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.368835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.369170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.372658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 306/612] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:44.377702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.395337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:44.412353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:44.413330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:44.414198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:44.419032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 307/613] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:44.424891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.515355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100123d-1380-445a-98e4-874615f28f97 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.542055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.554945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.555854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.556145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.556428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.557365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:44.557705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:44.559807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:44.569061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 307/614] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:44.576221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.658887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a0570-9ab4-4573-ac1b-ba57e0c2d6e7 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.699011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.702305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.703054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.703565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.703918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.705249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:44.705742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:44.707681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b68fe-f0c3-416e-83ff-eb5681b354ef None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:44.719142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 308/615] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:44.721107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9b09-b0ef-4042-bddb-3e07ae9f5e4b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.721475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9b09-b0ef-4042-bddb-3e07ae9f5e4b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.722021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9b09-b0ef-4042-bddb-3e07ae9f5e4b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.723736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfd55b-af20-4b03-915b-67f62871d80d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.727198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 309/616] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:44.728060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d9b09-b0ef-4042-bddb-3e07ae9f5e4b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.766497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.767634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.768026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.768407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.780105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:44.781776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-bcd26858-b25b-440e-a868-d2086d74a461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:44.800698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:44.801620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:44.802492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:44.802492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:44.802492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:44.802492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:44.802492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 308/617] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:44.803865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.804221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.804574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.824295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:44.826157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:44.843952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d9b09-b0ef-4042-bddb-3e07ae9f5e4b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:44.846364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.881037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:44.881828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.882166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.882441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.894346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:44.895888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:44.907224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 309/618] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:44.908343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:44.914302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:44.915218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:44.920606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:44.920606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:44.920606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:44.920606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:44.920606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 310/619] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:44.922717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.975502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.051548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f037847-9230-4013-827a-6d1a0154ae13 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.051746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14d072-b481-47f2-87b4-994431f217a4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.089535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.090389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.090862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.091160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.092217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:45.092535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.094259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.098060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.098986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.099496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.099968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.101450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:45.101977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.104115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.112981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 311/620] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:45.114205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.122942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 310/621] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:45.123869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.135730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.228444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b8c13-8116-4e4d-a5e1-a01525add52c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.246266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34cb05-0432-4bed-8deb-322b4b5ffa13 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.250621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.268059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.268841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.269245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.269623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.281567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:45.288094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.289088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.289519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.289954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:45.291453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:45.292571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:45.293829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:45.299252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 312/622] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:45.300107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.301803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.303726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:45.320367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:45.321408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:45.322370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:45.325638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 311/623] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:45.326673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.384516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.410063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0720c-0de8-4116-ad03-1a666116b982 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.439039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba92d-853a-46d8-8260-80b689835396 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.446776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.447381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.447668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.447946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.456136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.457691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:45.473395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:45.474342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:45.475311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:45.479245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 313/624] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:45.480296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.488800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.489485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.489773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.490053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.490818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:45.491156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.492655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.503638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 312/625] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:45.503638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.533074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.594077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02a190-29aa-4c20-934e-e85ee0f20be6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.611374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787029c4-9739-4eda-ac1d-d6eece0fd3bd None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.635720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.637195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.637964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.638455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.640197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:45.640996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.643316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.647378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.648873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.649552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.649886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.650167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.651295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:45.651611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.653776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.668878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 314/626] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:45.671095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.678443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 313/627] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:45.678443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.771456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.788782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b115f36-9be0-4b0e-a5a0-d333b6c55d2e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.820896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7d308-9c64-4440-8c3b-ac814e3c8ebe None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.834373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.835992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.836492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.837219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.857442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.858265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.858612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.858956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.870046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.872560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:45.896196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:45.897185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:45.898100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:45.898100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:45.898100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:45.898100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:45.902853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 315/628] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:45.904581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.935064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.937503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.960125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.036756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3dd38-0c75-4592-9f9d-4ec8c5678d23 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.039414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.057052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 314/629] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:46.058349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.089081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.090304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.090797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.091313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.102687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.104475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:46.123296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.124220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.125105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:46.125105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:46.125105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:46.125105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:46.127042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 316/630] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.132455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.187338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118af7-ce49-4c10-b2ea-0c5139251173 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.222675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.223385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.223673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.223953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.224677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:46.224989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.226368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.237204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.241928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 315/631] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:46.241928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.245108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941da07-215a-4ae8-a039-92def22c15bf None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.281345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.282452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.282738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.283016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.284204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.293647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.296705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6237722e-9c61-44c4-a8c4-5b0879ccf9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:46.313261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.314342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.315378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:46.315378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:46.315378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:46.315378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:46.315378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 317/632] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.316105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735d31-1da2-4d5f-a270-a20c916745cc None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.316393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735d31-1da2-4d5f-a270-a20c916745cc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.316670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735d31-1da2-4d5f-a270-a20c916745cc None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.317938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 318/633] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:46.319241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af984-1b99-4324-9671-a347cd36f0b8 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.319454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af984-1b99-4324-9671-a347cd36f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.319721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af984-1b99-4324-9671-a347cd36f0b8 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.320937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 319/634] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:46.321852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af984-1b99-4324-9671-a347cd36f0b8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.344498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebfd65-2934-4791-9fd4-8f5b5e085360 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.377732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.378324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.378611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.378891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.379645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:46.379956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.381404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.393490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 316/635] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:46.394283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.426968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af984-1b99-4324-9671-a347cd36f0b8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.461902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.462685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.462969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.463270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.470183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.471648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a0458f7d-a353-4f5c-a3dc-3725dd12b800 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:46.486204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.487268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:46.488211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 320/636] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.488784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.488784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.489219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.496037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de356-4385-4899-a29b-efcd3a2a3505 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.499150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.505442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:46.506635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:46.509376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.531673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.532335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.532624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.532906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.533632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:46.533947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.535424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621a9a9-8057-4a5e-951c-14e91e4090f0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.547348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 317/637] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:46.548563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.548854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.549133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.565656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:46.566868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:46.576668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 321/638] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:46.577498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.640638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 318/639] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:46.641395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.676536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d260be-cfd9-4b46-b713-a26d3b831915 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.709882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.710667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.710949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.711253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.717900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.719360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.721771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:46.734021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.737909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 322/640] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.738963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.740246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.747509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676650f-e04e-4e6f-a161-ecdf0b299891 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.780619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.781472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.781959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.782573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.783521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:46.783840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.785317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.803327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 319/641] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:46.803982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.838785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf5c5-7e58-4d49-bcf6-410d011f064a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.872788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.873581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.873862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.874135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.880815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.882293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:46.896630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.897650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:46.898577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:46.903133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa41ad-b93b-4eb5-8a96-db0b9d319c4d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.903396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 323/642] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.904240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.939958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.940797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.941135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.941422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.952923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:46.960808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:46.961856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:46.962794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 320/643] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:46.963658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.970702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:46.978846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.010386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1e3f-5729-4079-9684-a53aee54e469 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.043535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.044113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.044393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.044666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.054589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:47.061849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eb62f-d136-40f6-8483-709379b79589 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:47.062946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:47.064032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 324/644] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:47.065031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.098310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.098972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.099298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.099609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.100371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:47.100684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.102182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.113463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 321/645] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:47.114337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.163324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399ad1e-65eb-4144-b74b-bd281836e138 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.196186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.196839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.197122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.197395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.217049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632ed0-dd8f-42ca-add2-685da44db9f3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.251549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.252215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.252503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.252780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.253558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:47.253867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.255320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.266806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.267470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 322/646] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:47.268257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.268369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.280364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.303695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.367773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9482c1-cefa-4403-87a9-18ed1b10de37 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.378650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 325/647] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:47.379516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.397156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.403030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.403241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.403341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.403433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.403720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.404473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:47.404785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.406248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.414179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 323/648] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:47.414943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.480990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324262-3c8f-4a40-874c-7e5f01bdf767 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.514100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20ba3-0d3d-4aee-a3d3-cc2c7c2d7e6a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.514803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.515507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.515798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.516073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.516813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:47.517122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.518472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.526509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 326/649] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:47.527396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.548461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.549355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.549642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.549925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/7e631e04eeba421885436f9f2a1330b3/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.564977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=7e631e04eeba421885436f9f2a1330b3, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.566539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.610233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.627504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a992a7-8fac-4ece-a290-b99a481b319d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.644286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 324/650] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:47 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/7e631e04eeba421885436f9f2a1330b3/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:47.645166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.663706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93a74-279a-4784-9593-18253340a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.664449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93a74-279a-4784-9593-18253340a14b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.664678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93a74-279a-4784-9593-18253340a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.664956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93a74-279a-4784-9593-18253340a14b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.665809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af93a74-279a-4784-9593-18253340a14b None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:47.666134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af93a74-279a-4784-9593-18253340a14b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.667650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af93a74-279a-4784-9593-18253340a14b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.674954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 327/651] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:47.675858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93a74-279a-4784-9593-18253340a14b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.719460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.800893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97544eda-ad76-4304-8103-4b1b6ad5522c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.830047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93a74-279a-4784-9593-18253340a14b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.856100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.857227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.857664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.858081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.874326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.875562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.876370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.876654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.876776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.876923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.889305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.889943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.897665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:47.904297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:47.905464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:47.906514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:47.906514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:47.906514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:47.906514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:47.908409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 325/652] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:47.909337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:47.914013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:47.915063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:47.916095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:47.916095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:47.916095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:47.916095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:47.916095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 328/653] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:47.921109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.041350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.049368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc5bee-e1e5-4671-acd0-954e484bfbb8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.056321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba45e1d-38c4-4020-ab08-f457a3f0b831 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.094903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.095996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.096284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.096562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.099797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.100469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.100752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.101028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.101756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.102065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.103529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.108583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.110111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.119200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 329/654] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.120090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:48.133202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:48.134115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:48.135003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:48.135003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:48.135003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:48.135003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:48.135679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 326/655] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:48.137115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.204559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.260414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1d706-0c92-4a11-936c-65a5f484816e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.279907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e93d3a-888b-4e1d-b8b9-e19e9c422519 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.306952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.308005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.309114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.309753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.310759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.311337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.313094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.318284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.319087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.319402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.319684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.327648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.329006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 330/656] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.329197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.330593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:48.344744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:48.345686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:48.346572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 327/657] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:48.347639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.359745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.444471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6939115-bdb1-4cd1-9bbb-9b9c8a2d1ef0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.457089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f0fd3-6a89-407c-98bf-4e9c973e2d90 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.463804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.483045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.484175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.484703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.485175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.486126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.486748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.488492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.495323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.496159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.496444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.496725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.503484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 331/658] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.504549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.505825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.507559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:48.522970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:48.524014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:48.524986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 328/659] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:48.525594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.602412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.621625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138296c-f16f-4929-a123-122b4480ef40 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.638917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd866cb1-6ed5-4ec0-81b9-058024bfe1ba None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.658715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.659616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.660043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.660483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.661518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.661986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.663657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.676160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.676966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.677248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.677522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.678931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 332/660] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.679905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.691272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.692501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:48.708215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:48.709338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:58:48.710529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:48.713893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 329/661] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:48.715321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.726952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.792971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8428a9c-1b26-41c2-af35-838ac4ea93b1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.821790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b8023-09a0-40b5-b962-6dc5c4dd3119 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.832213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.833541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.833972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.834397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.835404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.835951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.837606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.850932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.861240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.861478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 333/662] 10.222.0.59 () {56 vars in 1137 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.861912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.862199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.862886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:48.867889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.871876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=bbfd6e827b9e49949cd6543edc89bbaf) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:48.871876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.871876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.882824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 330/663] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/role_assignments?user.id=bbfd6e827b9e49949cd6543edc89bbaf&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:48.884306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.012647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.019509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cba04-45e7-4e50-aa9c-a5fb7df543fb None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.067795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.068742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.069017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.069289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.070134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:49.070441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.071891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.095788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 334/664] 10.222.0.59 () {56 vars in 1139 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:49.102936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.121082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a540e-5963-4b49-bf5f-d811e69b539d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.165208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.166194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.166675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.167201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.225927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d233-8e0a-45db-8d94-823baad41486 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.237764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.249500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.251414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.262542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.263819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.264053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.264331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.289717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.341040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.342339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.382667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 331/665] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:49.389467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.481922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 335/666] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:49.484731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.504500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.521055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8e46-f71a-40e2-a9ac-ca7926c86305 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.567673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.571463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.572931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.573736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.574987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.598652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.600476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:49.634879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:49.635843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:49.637488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:49.637488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:49.637488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:49.637488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:49.654089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 332/667] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:49.656777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.670958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeed1c-a05e-4e9d-9cf0-03537a89de0c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.736857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.737904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.738542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.738960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.763827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.765539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:49.809910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:49.811016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:49.812123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:49.812123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:49.812123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:49.812123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:49.814831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 336/668] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:49.819570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.859069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d5fb4-d8b5-40b2-844e-5c75da9fb003 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.907941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.909217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.909711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.910205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.911422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:49.912529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.914468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b110b-6acb-4037-be67-4ffb050777a2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.932265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 333/669] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:49.934067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393e376-d2c7-47a5-9ee1-348f86ce14d9 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.934608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393e376-d2c7-47a5-9ee1-348f86ce14d9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.935097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393e376-d2c7-47a5-9ee1-348f86ce14d9 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.941157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 334/670] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:49.941157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393e376-d2c7-47a5-9ee1-348f86ce14d9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.947341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.955738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87756c5b-841e-4e2d-ab06-1848fab720b8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.963188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.993510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.994176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.994460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.994737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.995502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:49.996519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.001290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26321a91-49c9-4dd8-95d9-7dfad7dd01ea None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.012683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 337/671] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.014557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91287-4112-4f02-bbef-50ac02ef3bef None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.014989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91287-4112-4f02-bbef-50ac02ef3bef None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.015324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d91287-4112-4f02-bbef-50ac02ef3bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.032218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-39d91287-4112-4f02-bbef-50ac02ef3bef None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:50.033420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-39d91287-4112-4f02-bbef-50ac02ef3bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:50.059625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393e376-d2c7-47a5-9ee1-348f86ce14d9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.098885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.099976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.100393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.100823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 338/672] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dc2fe-5058-42bf-83f2-617f97f46a03 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dc2fe-5058-42bf-83f2-617f97f46a03 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739dc2fe-5058-42bf-83f2-617f97f46a03 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 339/673] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:50.115155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dc2fe-5058-42bf-83f2-617f97f46a03 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.117268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.118821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-64294449-fcf3-4933-8ea3-10e3be25e555 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:50.137929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:50.138926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:50.139855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 335/674] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:50.140884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.140884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.141074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.158734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:50.159949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:50.221254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.228106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.228741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739dc2fe-5058-42bf-83f2-617f97f46a03 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.239442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 336/675] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:50.241041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.264990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.265881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.266236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.266608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.273491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.274993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:50.289189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:50.290302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:50.291428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:50.292522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 340/676] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:50.293715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.360544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262fb01-106c-4f27-82a7-04b22bcfb205 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.399242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.400040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.400322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.400852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.401336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0024a-87e5-4a51-a86e-b0106d8b38e0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.412097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:50.420514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:50.421449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:50.422293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:50.426932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 337/677] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:50.427560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.438479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.439262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.439623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.439975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.440838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.441229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.442727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.455416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 341/678] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.456149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.474375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.492144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.541141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf131b95-3dd5-4565-983a-a0fdc3485d80 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.564470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01d0d-7024-4e79-820a-19842563b1cc None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.577165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.577959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.578243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.578521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.589237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:50.597650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:50.598593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:50.599433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 338/679] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:50.600756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.606522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.607228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.607518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.607795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.608593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.608908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.610356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.627388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 342/680] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.627388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.706322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.708524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd9da-ad33-4ad3-bf2c-a86b9dba2733 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.723581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.737761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae85f37-5b52-4796-b9c1-5bf2d3024a64 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.743906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.744679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.745065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.745527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.746377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.747133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.748804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.762256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 339/681] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.762256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.778226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.779135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.779523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.779901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.780832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=c86f8e914f3f4312a60791af443b2f5d) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.781245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.782775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.788682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 343/682] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/role_assignments?user.id=c86f8e914f3f4312a60791af443b2f5d&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.796476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.880404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3106b9b-85e5-4206-8cfd-88238edab874 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.926988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.927868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.928435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.928818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.930181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.930899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.932639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.940106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581406-76a5-4cb9-850e-8b60468566de None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.945977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 340/683] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.946944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.983951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.984616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.984898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.985172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.986003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=0a463c3c3f974bae83a6f4b21729f503) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:50.986313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.988443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.995035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 344/684] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/role_assignments?user.id=0a463c3c3f974bae83a6f4b21729f503&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:50.996691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.001223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.015114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.102390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfe9d8-e913-4d1d-9d00-dbd8714b9017 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.148908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04099fe-b301-4367-b8ee-50cb55c51dd4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.170663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.171500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.171785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.172062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.191020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.191020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.191320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.191320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.191320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.192045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.204884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.206495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:51.206772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:51.207844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:51.208893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:51.208893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:51.208893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:51.208893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:51.211234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 341/685] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:51.213882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:51.229022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:51.230170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:51.231309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:51.231309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:51.231309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:51.231309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:51.233006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 345/686] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:51.238493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.303110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.314536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.325714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c06c2-657f-42e8-bc2c-cca5de603302 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.353823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef479-c3b9-45dc-881f-82c08daca53a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.364725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.365405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.365690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.365966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.366754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:51.367062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.369162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.386689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 342/687] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:51.387461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.395924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.396937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.397443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.397886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.402640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:51.402640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.402640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.416547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 346/688] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:51.422879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.499015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f77e1-bd31-4656-a1c1-d421adba5ab9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.538272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.539047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb7a6-e35a-4189-be6c-97a8434cc1ae None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.539204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.539419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.539694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.548591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.550049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.559546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-af547ab3-6f87-4f50-99a8-536463143a47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:51.565942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:51.566951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:51.567939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:51.569801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 343/689] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:51.573300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e2c9-32e3-46b4-88e6-51aec3313de9 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.573743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e2c9-32e3-46b4-88e6-51aec3313de9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.574147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679e2c9-32e3-46b4-88e6-51aec3313de9 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.575670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 344/690] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:51.579292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679e2c9-32e3-46b4-88e6-51aec3313de9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.580609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.581422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.581709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.581990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.590032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.590676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.591740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-d6e2f01e-b14b-44c8-a839-c8ef6d84b313 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:51.615729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:51.616752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:51.617795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:51.621916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 347/691] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:51.623564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.624081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.624576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.645610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:51.646818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.712223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679e2c9-32e3-46b4-88e6-51aec3313de9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.723168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 348/692] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:51.723168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.771029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.771858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.772146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.772423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.773263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:51.773573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.775001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.801127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 345/693] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:51.803413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.850982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243438fa-e696-42e1-aa4f-a8c5be50672b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.861254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.884604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.902988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.904071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.904583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.905113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.906221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:51.906761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.908463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.925302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313dbe-2ed8-4270-b5ab-ff17365f6bd2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.931373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 349/694] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:51.932581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.967953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.969121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.969467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.969810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.983687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.985884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:52.008399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:52.009532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:52.010633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:52.010633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:52.010633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.010633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:52.011874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 346/695] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:52.017854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.045354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e937c-9f5b-4541-8be7-ccc54c3bfd3a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.086528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.087667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.088181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.088674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.139523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.147254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d69a6-d8ff-49a6-a8d4-1b818a93435e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.162661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.164570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.172799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.203055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.203748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.204032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.204702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.281691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 350/696] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:52.282883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.297129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.299322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.351730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.403085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38171e8-c4c9-4b38-bb20-fed1b99d2eb8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.420833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 347/697] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:52.422208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.446858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.451629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.452772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.453213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.453780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.455083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:52.455685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.457635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.477341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 351/698] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:52.478233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.540182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acd0ed-5378-4346-b702-b08dbf7e22f5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.578975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.580302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.580810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.581400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.593246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.595210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.604132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8616cf-ba28-46af-a847-fe045704fb65 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:52.615151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:52.616099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:52.617009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:52.617009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:52.617009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.617009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:52.624296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 348/699] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:52.625400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.630399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.648087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.649718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.650192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.651141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.668741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.670916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:52.694183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:52.695215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:52.696102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:52.696102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:52.696102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:52.696102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:52.700915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 352/700] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:52.701962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.708793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.748259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24f2a1-05ed-4e4e-95f1-32f07f7ffd90 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.797000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.797594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.797874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.798154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.798860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.800340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.817575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5534bbe-45a1-4fbf-ba5a-f4147cdd6652 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:52.853526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:52.854343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:52.854639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:52.855013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:52.855864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:52.856183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:52.857673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:52.875291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 353/701] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:52.875291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:52.913398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:52.916043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 349/702] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:58:52 2022] POST /identity/v3/users => generated 254 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:52.917472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.028963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe51c4-2cf6-4046-9490-d503e97e32a4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.051053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef330f1-f06f-4290-b5d9-6de5a68b19bb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.082302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.083935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.084376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.085343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.086586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.087714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.088346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.091151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dacc17-66b7-4960-821f-ce73a2a885fb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.095189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.096094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.096445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.099024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.104949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.107240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.111258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 354/703] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:53.113196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef5a2-b904-47b1-b065-2a70ad9413de None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.114632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef5a2-b904-47b1-b065-2a70ad9413de None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.115336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef5a2-b904-47b1-b065-2a70ad9413de None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.117615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 355/704] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:53.118695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef5a2-b904-47b1-b065-2a70ad9413de None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-d766d5d8-1577-4936-995c-3d4669a20675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:53.135620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:53.137525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:53.138666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:53.140401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 350/705] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:53.141752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.144343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.144343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.171945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:53.173182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:53.241390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.248818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef5a2-b904-47b1-b065-2a70ad9413de None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.283187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 351/706] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:53.284634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.286064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.286866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.287171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.287453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.296546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.298062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:53.313238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:53.314366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:53.315545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 356/707] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:53.323169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.413788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a97be-6e3b-43a3-bbd9-6bd87eb466e6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.432442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.452196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.452922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.453216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.453498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.453982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c8e1-db72-4abe-8231-6f5833c6fef1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.454295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.454613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.456214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.471073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 352/708] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:53.472771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.500403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.501230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.501642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.501931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.502885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.503229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.505407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.523213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 357/709] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:53.524348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.545971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.592988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4647b-4e3f-4d69-93f0-890fa25587a0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.630665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.631620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.631901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.632269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.632926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.634403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.638347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459a7f-a28d-4596-a86a-f8f1bd6db94b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:53.649314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:53.650719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:58:53.652562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:53.654804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 353/710] 10.222.0.59 () {56 vars in 1129 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:53.655774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.678305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.679335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.679793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.680204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.681169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=7e631e04eeba421885436f9f2a1330b3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.681725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.683344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f335-7daa-4c49-b6b9-c00d9570a8fc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 358/711] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/role_assignments?user.id=7e631e04eeba421885436f9f2a1330b3&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf87-37e3-40f9-90fc-5f186cdaedc7 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf87-37e3-40f9-90fc-5f186cdaedc7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf87-37e3-40f9-90fc-5f186cdaedc7 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 359/712] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:53.695156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdaf87-37e3-40f9-90fc-5f186cdaedc7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.715714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.800930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:53.804175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64796efb-7bfe-4418-95b2-106a7a7efa7f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.822826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdaf87-37e3-40f9-90fc-5f186cdaedc7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:53.859423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.860017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.860304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.860582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.861424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.861738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.863276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5619bfd-8c58-446b-958e-1db4ff76bca4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.867021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:53.867723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.868007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.868281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.868988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:53.869290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:53.869578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 354/713] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:53.870998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:53.871175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:53.872274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:53.872562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:53.876803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 360/714] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:53.883600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:53.891563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:53.892820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:53.975024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 355/715] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:54.002406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:54.023391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10bcd-09ab-436e-ac8f-a6f3a3df05c8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:54.048318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:54.071885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:54.072617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.072900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.073225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.075874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:54.075874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:54.076376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:54.086864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 361/716] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:58:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:54.136654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247df349-4326-4d42-b57b-da51ba6e1f7a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:54.190294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:54.191528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.192010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.192478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.193357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:54.195017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:54.208719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:54.209782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:58:54.210604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:54.215312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 356/717] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:54 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:54.220866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:54.239578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:54.353846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a18d9-0e8c-4943-a933-90da0a9c53fa None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:54.398248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:54.399331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.399833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.400572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.401499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] RBAC: Adding query filter params (name=network) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:54.402871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:54.404582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6905a8-0844-4612-b553-8da2331eef19 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:54.411264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 362/718] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:58:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:54.416684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:54.576742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:54.580412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b08e84-d963-4e6d-97cf-23e94f8befd2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:54.619146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:54.619824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.620111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.620501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.621243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] RBAC: Adding query filter params (type=network) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:54.621562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:54.622945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d29ea-b2fe-426c-a95e-5d4c5295bb92 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:54.629001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 357/719] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:58:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:55.314591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd76bee-0e11-4ad5-aa6e-4f212b2d01d4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.315142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd76bee-0e11-4ad5-aa6e-4f212b2d01d4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.315142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd76bee-0e11-4ad5-aa6e-4f212b2d01d4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.316786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 363/720] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:55.332821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.333304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.333420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.355242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:55.356479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:55.443889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 358/721] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:55.476179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd76bee-0e11-4ad5-aa6e-4f212b2d01d4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:55.695627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:55.735780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd76bee-0e11-4ad5-aa6e-4f212b2d01d4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:55.783825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:55.784625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.784904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.785172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.799910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:55.808218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:55.809168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:55.810070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 364/722] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:58:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:55.817243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:55.946510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:55.953967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdffdc8-d49a-4503-9f21-8fd10116ff10 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:55.995835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:55.996527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.996836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.997130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.997896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:55.998207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:55.999622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:56.008481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 359/723] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:58:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:56.014929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.187192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66875-a19e-4066-a982-fb9fa02e9db5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:56.231495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:56.240239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:56.241077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.241365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.241647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.256412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:56.257872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-51d3088c-db40-4c05-a91c-27543709e2bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:56.282605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:56.283662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:56.284560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:56.284560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:56.284560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:56.284560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:56.284560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 365/724] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:56.291630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.415334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda123d-963f-4838-938f-970196b2c1d6 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:56.458138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28154f3-e17b-4608-b018-574fff2a03a0 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.458913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:56.459358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28154f3-e17b-4608-b018-574fff2a03a0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.459612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.459900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.460180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.460495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28154f3-e17b-4608-b018-574fff2a03a0 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.460914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:56.461224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:56.462702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:56.462918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 366/725] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:56.468784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50779d-0f14-4907-aafb-06327b2a72a7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.469351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50779d-0f14-4907-aafb-06327b2a72a7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.469831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50779d-0f14-4907-aafb-06327b2a72a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.481021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 360/726] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:56.487680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.498336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7f50779d-0f14-4907-aafb-06327b2a72a7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:56.499778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7f50779d-0f14-4907-aafb-06327b2a72a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:56.521834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:56.620362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad662f4-e19e-4736-8bb1-627fcdc0228c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:56.632931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 367/727] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:56.635497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad0def-72ac-43d3-8e48-d1cc5b6da4b9 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.636215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad0def-72ac-43d3-8e48-d1cc5b6da4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.636892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad0def-72ac-43d3-8e48-d1cc5b6da4b9 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.638796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 368/728] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:56.653970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.654566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.655052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.664024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:56.664916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.665204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.665530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.673782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:56.674170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:56.675544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:56.675754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:56.692395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:56.693351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:56.694312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 361/729] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:56.694916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.767385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 369/730] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:58:56.768630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.777463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:56.801419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f949b-092e-48da-a16a-5d6085930d2b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:56.834337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:56.838290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:56.839316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.839607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.839888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.851004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:56.858909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:56.859975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:56.860823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 362/731] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:56.861511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.903779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fad75-8e34-4d18-b10f-f1552193894c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:56.946190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:56.947308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.948087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.948927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.949779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:56.950086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:56.951552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:56.969276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 370/732] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:56.970097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.993048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18852b-cc39-47eb-9b0c-a8f9244aebcb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.031799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.032515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.032801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.033083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.035703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.044295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e39293ad-894f-424c-9bd5-d9effb08addb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:58:57.052093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:57.053033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:57.053955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 363/733] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:57.055417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db81ea-a52c-4e98-988c-81ed1b23ca2b None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.055417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db81ea-a52c-4e98-988c-81ed1b23ca2b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.055417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db81ea-a52c-4e98-988c-81ed1b23ca2b None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.056800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 364/734] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:57.059242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db81ea-a52c-4e98-988c-81ed1b23ca2b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.085286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570f28d-2766-40d2-b4f0-adff2050097d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.086813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.127616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.128295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.128581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.128860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.129604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:57.129914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.131314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.140417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 371/735] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:57.141227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.171954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db81ea-a52c-4e98-988c-81ed1b23ca2b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.216533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.217323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.217607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.217881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.252980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be227b-f7ab-4f5b-b337-860a825d0030 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.280582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.286631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.287199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.287300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.287545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.287823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.288571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:57.288660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117925ca-7e83-4ae8-b891-82b19fa4b01a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.288882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.290230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07af9a3-1f28-48a5-9e53-6013bf7ba150 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.298400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 372/736] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:57.299532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.299821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.300095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.316468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:57.317701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:57.337932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.384468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 373/737] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:57.385400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.396908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 365/738] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:58:57.396908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1428918-3291-4756-a323-9e300284b660 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.396908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1428918-3291-4756-a323-9e300284b660 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.396908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1428918-3291-4756-a323-9e300284b660 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.399256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 366/739] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:58:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:57.400377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1428918-3291-4756-a323-9e300284b660 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.489859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2680f38-197f-44da-b19f-e604808122a3 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.498010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.504731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1428918-3291-4756-a323-9e300284b660 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.524670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.525721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.526004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.526278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.536283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.537784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.540233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.541015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.541298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.541574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.551924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.553386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:57.555392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:57.556298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:57.557201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:57.557201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:57.557201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.557201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:57.557201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 374/740] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:57.558487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.568806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-353424a1-6079-4d66-9d7d-f8fd4bb5ddf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:57.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:57.575866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:57.576741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:57.576741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:57.576741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.576741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:57.576741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 367/741] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:57.577183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.577518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.577719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.593675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:57.594872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:57.661207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268096d-32fe-4d94-b326-07726e6c8b10 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.661391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 368/742] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:58:57.662010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.695970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.696757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.697046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.697324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.708743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.710224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:57.726965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:57.728003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:57.728904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:57.728904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:57.728904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:57.728904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:57.728904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 375/743] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:57.729186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.747948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.768432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7e424-09db-4c5e-a376-d361a90dde6f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.790932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.803010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.805381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.805381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.805381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.805381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.806436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.832373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518a7-f2a6-4b84-a276-810947ed0890 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.865109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.865800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.866082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.866354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.867066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:57.867402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.868751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.877870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 369/744] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:58:57 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:57.877870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:57.894695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 376/745] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:57.894695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.003582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65b134-0e99-403b-80f6-c3a03802f461 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.004130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f164a9-1fab-4588-a772-4988e41dc1dd None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.027625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.043543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.043739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.044158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.044398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.044500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.044684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.044782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.044962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.045439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:58.045687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.045847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.047136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.047235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.048856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.060386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 370/746] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:58.063191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.096214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 377/747] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:58:57 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:58.097135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.175705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde50e4-be3f-44b6-9cb2-c6ace31de178 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.223643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.224558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.224958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.225309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.226141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:58.226619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.228383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.247638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 371/748] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:58.249100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.260211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1787-a598-4705-8e1b-f45428904bec None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.311908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.313743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.314912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.315214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.318795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.326446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.327948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:58.345412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:58.346393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:58.347375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 378/749] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:58.348455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.381687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2df158-572f-4cff-9374-c74c9dd7d9d0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.422890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.455037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.455934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.456192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.456475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.464886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.466365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:58:58.481818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:58.482806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:58.483786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 372/750] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:58.484385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.498789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859af66c-c24d-4722-8c30-c1181cfe8e11 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.544263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.545395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.545725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.546004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.556112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.558562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:58.577876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:58.578780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:58.579676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:58.582143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 379/751] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:58.583275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.619859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9acd6-fdaa-4b3b-9dc1-752ba8f17616 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.635046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.662144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.663222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.663795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.664359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.665599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:58.666441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.668931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.699182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 373/752] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:58.699182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.706294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.709376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a856349-d91a-4a1e-9a88-36a9e5bce853 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.754773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.756214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.756598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.756972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.758319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:58.758998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.761505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.789823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 380/753] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:58.789823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.839057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95a381-63e1-4f75-8e5e-dbb9cc548c73 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.885182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.887384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.887971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.888527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.889464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:58.890173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.892567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:58.909967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 374/754] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:58.911399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:58.912773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe43a26-62a3-4ff2-b81e-618b2b556a04 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:58.949229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:58.961083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:58.962417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:58.962792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:58.963183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:58.977029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:58.978623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:59.002618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:59.003621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:59.004607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:59.004607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:59.004607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:59.004607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:59.011582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 381/755] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:58:59.012667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.025565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:59.036730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f720-80b3-4163-8b46-bb78de4b680e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.078161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.078916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.079328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.079634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.120530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5ea8c-1717-4dd3-838e-781d1c4535f5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.155209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.156741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.156898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.157783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.158289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.158762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.159839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=54bf77c17b884999a745b47787217a5a) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:59.160361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.161976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.169049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 382/756] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/role_assignments?user.id=54bf77c17b884999a745b47787217a5a&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:59.170230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.197353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:59.255682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:59.284096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 375/757] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:58:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:58:59.284890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28330f-a010-49db-806b-2bad33b092f4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.289707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.380193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.380796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.381083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.381365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.382117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:59.382428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.383842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.409727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 383/758] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:59.415925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.573336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9151d-6e7a-4858-a9ea-e0e65143be04 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.589560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7210f6-c5d0-4f03-90cf-d5343c3e5b80 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.610347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.611562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.612076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.612539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.619739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.621380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:59.622864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:58:59.641886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:59.642826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:58:59.643763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 384/759] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:59.647912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.676432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:59.707980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.708654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.708943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.709224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.748049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.749515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.761342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51385-b86e-47d1-94c1-022a26b5d88b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:58:59.791161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:58:59.793203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:58:59.794095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:58:59.794095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:58:59.794095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:58:59.794095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:58:59.795261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 376/760] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:58:59.803036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.804851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.814892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.815583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.816060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.817097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:59.817621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.820734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.849802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 385/761] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:59.856235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.912732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe4d47-5797-4cc7-9117-6b824cc8d810 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:59.947487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:59.949192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.949679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.950167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.951083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:59.951609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:59.953184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:59.966122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 377/762] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:59.971103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.018847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.027904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.078257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4e91-8594-4cc0-911c-e76e236c7278 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.136992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64e58-db34-4277-835d-a85847851da8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.138550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.139676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.139968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.141102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.143161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=c101547390374884b75d95b637267e79, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.144608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.173039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.177290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.177644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.177991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.188947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.190405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-92b382f9-50e3-4a0a-8fe9-d5658c8ed6bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:00.205988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:00.206914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:00.208401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 378/763] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:00.208890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe8f-ee1c-405b-9dde-55c36607dd55 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.209140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe8f-ee1c-405b-9dde-55c36607dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.209376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe8f-ee1c-405b-9dde-55c36607dd55 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.210844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 379/764] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:00.215810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.216257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.216537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.218548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 386/765] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:58:59 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:59:00.227211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.253568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:00.256150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:00.380599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2acbe4-0322-4f14-b1ab-85460250ad76 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.395262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 380/766] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:00.396448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.425557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.426411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.426888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.427607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.428430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:00.429986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.431544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.433170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.448247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 387/767] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:00.456514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.473613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.569395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b698f-e5e1-484a-81b0-0edc7459318a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.584933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1f1c-bf2a-4206-9ee0-72d5b1b298c5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.609366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.610468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.610754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.611032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.611887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.613906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.623475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.624142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.624430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.624708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.625542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=7e631e04eeba421885436f9f2a1330b3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:00.625856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.627315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b29011-b6b0-451d-97f1-381fb8c79490 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-12c44d6c-a456-4bcf-9afb-f723e0966ac6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:00.628512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:00.629383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:00.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:00.635205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 381/768] 10.222.0.59 () {56 vars in 1129 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:00.637009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df634ad9-971c-4953-993b-4b4a66f70469 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.637296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df634ad9-971c-4953-993b-4b4a66f70469 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.637575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df634ad9-971c-4953-993b-4b4a66f70469 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.639049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 382/769] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:00.640693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df634ad9-971c-4953-993b-4b4a66f70469 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.643330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 388/770] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/role_assignments?user.id=7e631e04eeba421885436f9f2a1330b3&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:00.645367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.645660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.645941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.666252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:00.668066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:00.703365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.778920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 389/771] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:00.802686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.817472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.832947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df634ad9-971c-4953-993b-4b4a66f70469 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.880770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.881445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.881732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.882010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.882888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:00.883270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.884657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.890295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 383/772] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:00.896397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:00.977331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79ed95-f5a2-4ccc-84f8-651203c98a1e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.038615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1cfd-8a67-458d-a9e8-6a90cbae652c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.057245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.058791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.059078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.059382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] PATH_INFO: `/v3/services/network` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.060051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.061557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:01.085154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:01.086236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:01.087099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 390/773] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:01.087907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.087907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.089491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.089491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.089730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:01.090027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.091448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.101067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.101826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 384/774] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:59:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:01.110011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.162064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:01.163464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:01.230310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733364-ec09-4233-8f3b-a5d5097c1f65 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.267229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.268065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.268453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.268931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.269845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] RBAC: Adding query filter params (name=network) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:01.270262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.271798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.277983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 391/775] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:01.283007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.335006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424379d-3514-4141-baae-f9caf20e059f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.399365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.400284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.400567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.401056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.401897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] RBAC: Adding query filter params (interface=public, service_id=a5e8e0cca2834612bf89173b0e9ab90d, region_id=RegionOne) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:01.402209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.403616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.411501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:01.416880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46954-f887-471e-a263-d98dd0c2fc50 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.426732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 385/776] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/endpoints?service_id=a5e8e0cca2834612bf89173b0e9ab90d&interface=public®ion_id=RegionOne => generated 194 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:01.434295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.482529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.483188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.483478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.483762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.484470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:01.484924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.486408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.491811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 392/777] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:01.499253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.588476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:01.613666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e231a-26a4-4d55-99ea-109ec6f104c1 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.656015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.656790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.657181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.657466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.658165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.659587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa2402-1301-4cf7-9916-661e14b3ad40 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.660051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142f254-6b37-47b5-8ba9-85cf3d95f1d2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.665612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 386/778] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:01.667691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45c41-751c-4f75-b2a2-f436dc09fef1 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.667982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45c41-751c-4f75-b2a2-f436dc09fef1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.668266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd45c41-751c-4f75-b2a2-f436dc09fef1 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.670177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 387/779] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:01.677531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f374-12bb-4531-b19a-141da9187884 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.677915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f374-12bb-4531-b19a-141da9187884 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.678553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543f374-12bb-4531-b19a-141da9187884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.706686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.707515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.707925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.708308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.709128 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] RBAC: Adding query filter params (interface=public, service_id=da8b12d22c5049de81453a2b6c506b0c, region_id=RegionOne) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:01.709442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.710853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.720376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5543f374-12bb-4531-b19a-141da9187884 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:01.721431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 393/780] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/endpoints?service_id=da8b12d22c5049de81453a2b6c506b0c&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:01.721615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5543f374-12bb-4531-b19a-141da9187884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:01.728171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.763291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:01.849079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45b4c-8243-4049-bd5e-ad4ad6df6463 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:01.856451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 388/781] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:01.886904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:01.887619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:01.887903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:01.888181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:01.888874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:01.890247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:01.892751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543f374-12bb-4531-b19a-141da9187884 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:01.896441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 394/782] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:02.017346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.038988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543f374-12bb-4531-b19a-141da9187884 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.081926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.082738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.083023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.083330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.096676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:02.105446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:02.106495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:02.107409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 389/783] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:59:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:02.113374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.214922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.229505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf59190-b7e0-460a-99c7-923f469639bb None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.270446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.271064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.271376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.271697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.272462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:02.272772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:02.274146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:02.284075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 395/784] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:02.289623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.423649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0660e9-750e-4e24-b46e-3c07af4f4955 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.467810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.468615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.468897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.469174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.481425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.487728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:02.489430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:02.509702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:02.510675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:02.511611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:02.511611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:02.511611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:02.511611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:02.515474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 390/785] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:02.520218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.638491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c4bb-63be-4973-8e78-962f1f43b448 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.674730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.675521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.675716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.675998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.676734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:02.677049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:02.678499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16879036-fcd0-4f44-b5dc-6773e69c26b7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:02.694895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 396/786] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:02.700265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.786339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9f9f-845f-411a-8c67-d3a190d11fe3 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.786538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9f9f-845f-411a-8c67-d3a190d11fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.786781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd9f9f-845f-411a-8c67-d3a190d11fe3 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.788371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 397/787] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:02.794426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.794781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.794982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.828271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:02.829580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:02.832769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.840111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd0fe-0830-412a-9981-b7a884440772 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.894093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.894835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.895138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.895427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.908263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:02.909724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:02.941122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:02.942158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:02.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 391/788] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:02.950251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.958063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 398/789] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:02.996260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.024453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.086602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51472c-b1c8-479e-8e20-7227eccccfbe None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.147575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.149286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.149829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.150111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.151051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:03.151494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.152950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.200955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 392/790] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:03.206192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.212270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3e8a5-9658-4f30-b2b4-243e8d5db0d4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.277273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.278079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.278378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.278648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.287041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.290783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-40dbff5a-0919-4ddc-805d-22fb24875730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:03.299557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:03.306688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 399/791] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:03.308177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115690-cbfe-40fe-9dfb-bb8b7524e0a4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.309078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115690-cbfe-40fe-9dfb-bb8b7524e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.309078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80115690-cbfe-40fe-9dfb-bb8b7524e0a4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.309078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 400/792] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:03.309078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80115690-cbfe-40fe-9dfb-bb8b7524e0a4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.353731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebc8e7-e055-4db0-b316-b49ef43d48e3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.365760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.394264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.395334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.395763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.397235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.415229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.415229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-04547242-5d22-4645-a403-0aabc6a4ca2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:03.429197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:03.430102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:03.431284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:03.431284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:03.431284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:03.431284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:03.435893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 393/793] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:03.437434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.437725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.437879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80115690-cbfe-40fe-9dfb-bb8b7524e0a4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.438021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.456839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:03.458096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:03.483565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.484505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.484863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.485473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.486520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:03.486833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.488721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.503236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 401/794] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:03.504177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.536066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 394/795] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:03.536931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.619583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.631934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eee526-6673-4b1d-af15-43f6e4596473 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.634499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.644327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf8c50-1656-472e-b08d-8a52bd6e2a24 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.668250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.668839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.669124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.669400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.670127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:03.670437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.672173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.687208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 402/796] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:03.688777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.690347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.691460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.691753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.692035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.705668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.707581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:03.732941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:03.733901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:03.734841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:03.734841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:03.734841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:03.734841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:03.738936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 395/797] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:03.739880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.815535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e69eb-df44-462e-af4a-49e0c483fff5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.844658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:03.859725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.860545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.860885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.861538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.871332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bd132-382b-4a7b-a814-dd68375f4fad None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:03.879993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:03.891001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:03.892005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:03.892839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:03.895197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 403/798] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:03.896275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.922544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:03.923308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:03.923600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:03.923883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:03.933908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:03.935428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:03.953776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:03.954712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:03.955640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:03.959269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 396/799] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:03.960301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:03.966974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.015370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeffe6-f7c3-43db-b942-e5a1ee391f17 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.051229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.052319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.052805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.053362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.054283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:04.054790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.056415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.070157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 404/800] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:04.071395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.099746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc466d38-5c84-413b-9a4d-32d45971c556 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.118723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.147832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.148519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.148828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.150259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.151553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:04.151909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.153367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.164351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 397/801] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:04.165521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.190412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1df10-b220-44d0-857d-fcdb1e76ec20 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.227197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.227858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.228138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.228541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.230096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:04.230411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.231854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e256180-ff82-4c31-a6a5-e2d8ae71dcc9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.239009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.249654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 405/802] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:04.249654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876697-11d5-484f-bda8-41fb8200a4c4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.249654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876697-11d5-484f-bda8-41fb8200a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.249654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876697-11d5-484f-bda8-41fb8200a4c4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.252212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 406/803] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:04.253036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876697-11d5-484f-bda8-41fb8200a4c4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.276630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9e18-e7d5-47f7-9169-7368fab1b3a4 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.313738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.314542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.314835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.315112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.323826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.325281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:04.341494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:04.343674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 398/804] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:04.344601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.368629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876697-11d5-484f-bda8-41fb8200a4c4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.371099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.409090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.409887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.410174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.410448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.420698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.422145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-1745ff9f-ef7e-4d18-ac3c-0340517f00af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:04.440007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:04.440867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:04.441751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:04.441751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:04.441751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:04.441751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:04.441751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 407/805] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:04.442594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.442882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.443180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.453019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f943e7-cf0a-40a4-bb33-e596df5848ee None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.460740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:04.461917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:04.477814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.487203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.488105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.488370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.488644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/54bf77c17b884999a745b47787217a5a/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.504123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=54bf77c17b884999a745b47787217a5a, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.505588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c939d-4f54-406d-86a4-d82c64937d11 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.529970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 408/806] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:04.530805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.559990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 399/807] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:04 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/54bf77c17b884999a745b47787217a5a/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:04.561706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688bd9-27b6-4cb2-b427-dc05c70d3229 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.562054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688bd9-27b6-4cb2-b427-dc05c70d3229 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.562344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688bd9-27b6-4cb2-b427-dc05c70d3229 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.564074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 400/808] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:04.569280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da688bd9-27b6-4cb2-b427-dc05c70d3229 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.621951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.640560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7582a1-8df4-4999-bf5b-2417d3c5a869 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.681420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.681689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da688bd9-27b6-4cb2-b427-dc05c70d3229 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.682755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.682986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.683292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.684192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:04.684507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.685988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.708417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 409/809] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:04.711244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.726600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.727352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.727645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.727926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.729175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:04.729488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.730884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4818e-82c3-4da8-870d-c092d73f102e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.743948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 401/810] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:04.746364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.746802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.747247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.764420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:04.765700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:04.794876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.829961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7ee99-4cf3-4278-9438-14ff40b56f29 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.839319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 402/811] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:04.839319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.877421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.878290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.878741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.879026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:04.879720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:04.881126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:04.894274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:04.895449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:59:04.896556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:04.900725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 410/812] 10.222.0.59 () {56 vars in 1129 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:04.901595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:04.911335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:04.946561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae24a4-4a56-42be-b415-9cbb41f60dc3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:04.982456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:04.983142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:04.983444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:04.983723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.013618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ccc3b-3379-408a-86e3-0547ec20aecc None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.021867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.051390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.052265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.052534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.052863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.059103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.060597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.061830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.064344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:05.080531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:05.081547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 411/813] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:05.082516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.154664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.179741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 403/814] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:05.180567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.198126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eed61a-17e2-480f-8c76-00a812bd57dc None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.235205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.235929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.236214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.236493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.237160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.238530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:05.250276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:05.251291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:05.252152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 412/815] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:05.253729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.254314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.292873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bfedc-bdcf-44e7-a259-f57f315628c9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.329126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.330131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.330525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.330810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.331558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.331873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.333253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.343370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 404/816] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.343370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.367560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a083-f71d-465e-9667-e0faffa95ba4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.397867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.403815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.404493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.404779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.405069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.405903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.406218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.407697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.426088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 413/817] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.426801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.456622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172b8f1-ce49-4840-957e-b46d31f32511 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.489721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.496674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.499244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.499244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.499244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.510134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.512086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:05.531420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:05.532321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:05.533335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:05.533335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:05.533335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:05.533335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:05.533335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 405/818] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:05.533608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.539281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9149-7753-4cf9-8eea-5d3a124c9dd8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.578151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.578821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.579111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.579433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.580140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] RBAC: Adding query filter params (name=network) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.580451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.581814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.586638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 414/819] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.587537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.643654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.644715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9ca87-ee1a-4e15-8d14-83dad0cb2b76 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.686454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.687277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.687574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.687852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.688586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.688900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.690338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.696539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 406/820] 10.222.0.59 () {56 vars in 1124 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.700385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de091-f9da-4c2e-ba6e-c731a5149935 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.702536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.739868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.740651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.740933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.741209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.751289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.753481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.754941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:05.773575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:05.774807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:05.776114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:05.776114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:05.776114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:05.776114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:05.777968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 415/821] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:05.779213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.814707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb2eb9-dd05-4e1d-be89-5a7fe1f58d00 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.853374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.854076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.854358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.854643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.855393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.855722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.857099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.874427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 407/822] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.874427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.892029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9aa21-f2c3-4b7e-910b-d291c81992d1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.895131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.927334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.928390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.928890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.929179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.930201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] RBAC: Adding query filter params (type=network) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:05.931079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.933335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.942657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 416/823] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:05.942657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.983405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5ce70-199b-453d-bcb3-9738bbb01c46 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.991409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.021790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.022457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.022745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.023024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.023911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.025752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.043390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf95175-46c8-41fc-8644-558d447f86da None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.077799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.078510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.078843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.079188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.079925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:06.080239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.081675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.093190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 417/824] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:06.093979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.112145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 408/825] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:06.124555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.129357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.232887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912c164-a497-46a9-94e2-36d66334485a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.276832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d6fa-7bbd-48fa-8d94-eb8d28669ae9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.287942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.289043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.289521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.290029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.298236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.299991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:06.315028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:06.315989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:06.316853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 418/826] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:06.317983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.328412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.328412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.328412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.328729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.329379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.330775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.398374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 409/827] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:06.399608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.441171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.457738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.482518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c85964-b0e5-4354-83ba-8ff91045ddf6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.577653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.579384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.579700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.580874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.596746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.598221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:06.618970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:06.620526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:06.621436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:06.621938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3054591-9e32-4b87-91f3-5333679d60b3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.625686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 419/828] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:06.626582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.673989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.674673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.674963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.675352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.676158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:06.676474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.677918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7af2-ba6b-451f-807e-646089ad8c4e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.693497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 410/829] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:06.693497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711d5a-eefc-4c68-ae69-3911c9d2fbce None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.693497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711d5a-eefc-4c68-ae69-3911c9d2fbce None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.693497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30711d5a-eefc-4c68-ae69-3911c9d2fbce None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.694913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 411/830] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:06.703035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.703359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.703628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.723459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:06.724668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:06.764511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35715b-f75e-4fda-ab9c-6042ff641246 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.800989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.801657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.801949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.802231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.803029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:06.803377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.804817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.808524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 412/831] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:06.809648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.817102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.821646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 420/832] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:06.827585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.836446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:06.927024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2402d-f41a-44f9-943e-702c0032d6b0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.935592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1240d-6816-4c1f-88cb-30d166318f23 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:06.966396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.967054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.967362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.967640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.968472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:06.968790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.970224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:06.976019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:06.983339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 413/833] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:06.983526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:06.983526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:06.983685 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:06.984091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/7e631e04eeba421885436f9f2a1330b3/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:06.985688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=7e631e04eeba421885436f9f2a1330b3, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:06.986714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:07.063640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:07.065657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 421/834] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:06 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/7e631e04eeba421885436f9f2a1330b3/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:07.115749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe9c6-9865-4a8d-8485-4b8222b7256a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:07.158072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:07.158910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:07.159261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:07.159510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:07.172152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:07.186122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:07.187182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:07.188034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 414/835] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 10:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:07.193452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:07.264523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:07.304614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76911d05-02d8-46b0-b149-a0c74890c2b6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:07.346993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:07.347702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:07.347987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:07.348265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:07.349087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:07.349398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:07.350773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:07.361029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 422/836] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 10:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:07.366561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:07.477140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c069b8e-a4d9-4841-b712-b1ae9a6213a2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:07.505998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:07.514078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:07.514884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:07.515191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:07.515475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:07.526496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:07.528046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:07.545876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:07.546830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:07.547960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:07.547960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:07.547960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:07.547960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:07.547960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 415/837] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:07.552708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:07.661963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a7578-5204-49ff-9428-fdfda100364f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:07.690288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:07.699280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:07.699952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:07.700235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:07.700514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:07.701237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:07.701553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:07.702920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:07.716549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 423/838] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:07.721946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:07.830977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13e8bc-443a-479b-98b0-237d0a910fae None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:07.867546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:07.868382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:07.868669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:07.868946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:07.877331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:07.878799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:07.893915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:07.896293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:07.898512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 416/839] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:59:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:07.903854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.007233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5afe3-2728-414c-abe5-5f99eb72032e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.044214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.044886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.045168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.045445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.046282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:08.046595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.048090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca95ff4-4325-49ca-a187-b1ddc3ee8f83 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.065983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 424/840] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:59:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:08.071615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.115626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.180573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26eabc8-33a1-42db-bf0a-a812098c7be8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.227144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.227814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.228103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.228422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.269107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8bb09-ee64-4361-9ba1-55c5bcfdc249 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.269422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8bb09-ee64-4361-9ba1-55c5bcfdc249 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.269631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8bb09-ee64-4361-9ba1-55c5bcfdc249 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.271427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 425/841] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:08.280975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6707-986e-4239-a145-42bb45274de5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.281278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6707-986e-4239-a145-42bb45274de5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.281515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c6707-986e-4239-a145-42bb45274de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.298638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-540c6707-986e-4239-a145-42bb45274de5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:08.299885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-540c6707-986e-4239-a145-42bb45274de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:08.310652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.313165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.347201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.370039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.383401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 426/842] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:08.401381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c6707-986e-4239-a145-42bb45274de5 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.441927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 417/843] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:08.448498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.530600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c6707-986e-4239-a145-42bb45274de5 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.554032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b548d60-37f0-4539-8ed6-d4395e395fcc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.577799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.579090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.579550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.580402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.581476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.583687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.592021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.592935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.593325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.593704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:08.598873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:08.600071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:08.601115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:08.604939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 427/844] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:08.608785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.609985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.612095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.614961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:08.638029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:08.638962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:08.639871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:08.639871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:08.639871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:08.639871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:08.644301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 418/845] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:08.645305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.661443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.734632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a51cc-3cd1-4db7-8ed5-d89e2f905f61 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.764487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd719ab-9be4-4ad1-9c80-d346375c0109 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.776713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.777788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.778291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.778764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.779700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:08.780223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.781836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.787226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 428/846] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:08.793152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.810417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.811104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.811605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.812061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.812809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:08.813124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.814818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.836049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 419/847] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:08.837372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:08.898514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.911896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572d42-31fc-4452-af6d-b60609c6743e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.913510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:08.948748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.949433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.949701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.949998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:08.950712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:08.951025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:08.952533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:08.954569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146e974-1202-4257-aa99-cb00bd0435a3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:08.963692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 429/848] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:08.997554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:08.998794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:08.999329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:08.999814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.007986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.010163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:09.026830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:09.027871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:09.028786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:09.035589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 420/849] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:09.037578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.153016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdb79d-8bd9-409d-89b4-d5f59584daa2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.198895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.199945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.200382 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.200822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.201762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:09.202220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.203835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6ef0a-8899-4ecf-a861-046406a720ea None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.216863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 430/850] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:09.222909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.235600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff7ea-0d75-48f8-828d-633626a4dc1b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.236075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff7ea-0d75-48f8-828d-633626a4dc1b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.236365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff7ea-0d75-48f8-828d-633626a4dc1b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.238008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 431/851] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:09.242974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1edb-6543-4c5e-9f26-181b08517ef7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.243305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1edb-6543-4c5e-9f26-181b08517ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.243592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf1edb-6543-4c5e-9f26-181b08517ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.261842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2aaf1edb-6543-4c5e-9f26-181b08517ef7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:09.263060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2aaf1edb-6543-4c5e-9f26-181b08517ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:09.284477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.334449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 432/852] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:09.336677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f3f8-f7e2-4eaa-a8c2-21b296bb2d8c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.337131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f3f8-f7e2-4eaa-a8c2-21b296bb2d8c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.338155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f3f8-f7e2-4eaa-a8c2-21b296bb2d8c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.338574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acbc-7f14-4c4a-b275-ae22b2fb6416 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.343943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 433/853] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:09.345495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.345779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.346052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.360142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.364610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:09.365799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:09.385399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.386216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.386579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.386929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.387816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=54bf77c17b884999a745b47787217a5a) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:09.388137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.389677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.399209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 421/854] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/role_assignments?user.id=54bf77c17b884999a745b47787217a5a&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:09.399209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.463782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 434/855] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:09.487491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.538504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf99b-81d7-4912-bc10-88218044a33b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.581159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.582353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.582858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.583169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.583843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.585289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:09.600205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:09.601208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:09.602040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 422/856] 10.222.0.59 () {56 vars in 1132 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:09.608843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.612860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.627350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5aa77-bfe1-4270-bd5c-b918916e78e7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.659526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.671419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.672697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.673190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.673750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.689986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:09.698753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:09.699784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:09.700599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:09.701244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 435/857] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:09.706702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.730433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc823-b115-497a-ba8d-d48f47de14e9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.765079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.765751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.766085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.766369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.767085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:09.767423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.768798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.777645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 423/858] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:09.783405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.851508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559edfd7-3ed3-4d8a-b97d-9d685242f6ea None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.895051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.896771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.897259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.897733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.898463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad221fa1-82b7-40ca-813b-34778dc7e7be None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:09.899152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:09.899675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.902077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.903756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.915042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 436/859] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:09.923171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:09.936996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:09.937161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:09.937707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:09.937994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:09.938271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:09.939076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:09.939534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:09.940943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:09.945884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 424/860] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:10.049065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b25ba9-ddbe-4d02-8822-bdcdb9e0699b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.087227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.088976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.089161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.089408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.101233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:10.102733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:10.121533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:10.122444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:10.123604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:10.123604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:10.123604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:10.123604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:10.123604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 437/861] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:10.128331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:10.248119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13f3ba-94f6-4bb6-89da-3d62af159437 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.291320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.291320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.291320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.291320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.291739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:10.291739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:10.291739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:10.303840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 425/862] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:10.316208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:10.385535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.445665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c38f-e51f-446f-810b-50f544bc10fb None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.508951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.510013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.510298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.510573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.519264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:10.520697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:10.538885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:10.540057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:59:10.541533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:10.547142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 438/863] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:10.552218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:10.599081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.677005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fd16e-9f73-4323-a51e-5760801ef612 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.722662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.723370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.723654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.723928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.724767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:10.725074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:10.726499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:10.768112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 426/864] 10.222.0.59 () {56 vars in 1135 bytes} [Thu Jul 14 10:59:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:10.773751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:10.899367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b88969-2f1b-4d03-be5b-c14196491710 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.921088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.935747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.936338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.936625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.936905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.007908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.009393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36a7b-20e4-43b0-9825-ac3cc170055f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.110711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 439/865] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:11.120349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.203240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bad06-01ab-4207-bf33-fcada20e323b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.203240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bad06-01ab-4207-bf33-fcada20e323b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.203240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bad06-01ab-4207-bf33-fcada20e323b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.204282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 440/866] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:11.209288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.209579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.209862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.221764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.228132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:11.229342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:11.233687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76284474-4885-4c8e-bb3a-9f849c796470 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.245010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.271387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.272849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.273299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.273738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.285454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.287249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:11.308958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:11.309972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:11.310885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:11.310885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:11.310885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:11.310885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:11.311568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 441/867] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:11.313478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 427/868] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:11.321351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.330621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.427048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd0cf-6aa5-4a31-b2e3-94e72589d07a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.444222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7566-385f-4c15-8572-0fc7a884ebfe None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.462489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.463084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.463401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.463688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.464409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:11.464722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.466096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.482963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 442/869] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:11.484020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.486457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.487489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.488038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.488512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.489439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.491013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.497604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.558599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.575383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 428/870] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:11 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:59:11.601716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4eb5-789c-4b76-bfe6-bba97eb41090 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.649548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.650923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.651243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.651522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.659738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.661213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-de1b4f6d-d29e-4ea5-9cec-73dd2dbba466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:11.678095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:11.679082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:11.680010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 443/871] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:11.685777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.736673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57fb6-4ae8-40d2-8bfc-f68a04d323f3 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.737008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57fb6-4ae8-40d2-8bfc-f68a04d323f3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.737244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57fb6-4ae8-40d2-8bfc-f68a04d323f3 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.739003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 444/872] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:11.748647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.749312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.749790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.767387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:11.769109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:11.815035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d1e3-700a-4a45-acd2-81714e3c6d75 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.842957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.852049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 445/873] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:11.863166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.863166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.863166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.863166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.863566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:11.863566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.863566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.877063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 429/874] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:11.882921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.883081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.954649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.001982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61d1dd-2d2a-4990-8f08-d137ef339fe1 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.011377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161d4c5-155b-4f5f-9a29-51fd622f6069 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.042335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.043566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.043995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.044504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.052813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.053591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.053997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.054380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.055348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=7e631e04eeba421885436f9f2a1330b3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:12.055800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.057565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:12.057978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:12.066390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:12.067406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:12.068267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:12.075746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 446/876] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/role_assignments?user.id=7e631e04eeba421885436f9f2a1330b3&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:12.076104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 430/876] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:12.080529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.151869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.200610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfee37-e197-4e56-9a08-2f1e5a33e63c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.238150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.238888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.239202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.239491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.240236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:12.240546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.241914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d765deb-8dbf-43ee-9e04-e6887efb65cf None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.252056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 447/877] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:12.257697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.265829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009f90-040a-4c60-956e-851e8edcfb0e None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.266114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009f90-040a-4c60-956e-851e8edcfb0e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.266390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a009f90-040a-4c60-956e-851e8edcfb0e None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.268066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 448/878] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:12.272877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.273162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.273439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.292583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:12.293822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:12.368081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 449/879] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:12.385001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8270-289a-462a-8936-43de0f27b06d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.393593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.404022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.421972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.422772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.423054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.423356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.435070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.436555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:12.456068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:12.457003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:12.457888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:12.457888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:12.457888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:12.457888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:12.457888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 431/880] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:12.462809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.464243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.506631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072643a-4d7b-4427-8ea9-abaa4f9e24ee None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.543703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.544790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.545279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.545755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.546679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.548290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.574054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb581e-fb22-4d61-938d-c16f261ede3f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.607710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.608377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.608661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.608937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.609649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:12.609957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.611370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.623961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 432/881] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:12.629093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.660011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.683355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 450/882] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:12 2022] POST /identity/v3/services => generated 269 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:59:12.751151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c46e9-f430-42a0-b859-021173ee84c9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.801353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.802574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.803085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.803618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.813641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.815554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:12.833801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:12.834796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:12.835756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:12.838639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 433/883] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:59:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:12.843877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:12.953909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d2e3-2b5e-4ecf-b9b9-0a3cf4f680fa None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:12.967305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:12.989170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:12.989850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:12.990135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:12.990416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:12.991290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:12.991606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:12.993147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.011896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 451/884] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:59:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:13.017591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.108163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:13.125533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651c3ce-34f0-45ac-851d-5b560b2919ea None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:13.162369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:13.163456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.163953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.164419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.249095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:13.250977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.352257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:13.366772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 434/885] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:13.372983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.480400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb890c-d648-443e-be54-3feba7fe983e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:13.515901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:13.517037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.517482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.517929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.530273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:13.532003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:13.550577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:13.551525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:13.552435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:13.552435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:13.552435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:13.552435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:13.554313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 452/886] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:13.560700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.661787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0c940-dabc-4084-84c7-aa98a3b46c86 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:13.697960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:13.698908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.699357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.699790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.700645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:13.701095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:13.702595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09efd5-e8b6-4aff-a2e5-47b17d96ef23 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.715488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 435/887] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:13.721236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.748070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:13.774482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261280-8bc2-4ac0-bd9f-67adcb36e396 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.775483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261280-8bc2-4ac0-bd9f-67adcb36e396 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.775751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261280-8bc2-4ac0-bd9f-67adcb36e396 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.777345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 436/888] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:13.782637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.782918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.783215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.807511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:13.808713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:13.840102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102c8df-ae53-4613-acc7-bc53ef1f3a75 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:13.870648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:13.878650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:13.879465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.879877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.885139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.888378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:13.889924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.902049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 437/889] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:13.911148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:13.912124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:13.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 453/890] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:13.913483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.928872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.033740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe162416-a03d-4e2d-8594-b6f609a30ade None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.039914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.064725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94148-ca9a-4dfe-9ee4-9276ef9d7e8a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.079067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.079984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.080191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.080467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.081631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.081857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.083319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.101766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 438/891] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:14.102911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.113679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.114710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.114836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.115037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.115734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.117358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:14.130476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:14.131522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:14.132364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:14.137347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 454/892] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:14.138320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.149694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.226557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a487-819e-4e4b-ab26-d390463d461c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.254277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618718b-2b90-4365-9fcd-f2e6cd6b580c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.262511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.263238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.263523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.263797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.264656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] RBAC: Adding query filter params (role.id=5c09a21ee6d942068400f80f686e0132, scope.project.id=86229daf0943484393a30086a3139985, user.id=54bf77c17b884999a745b47787217a5a) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.264975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.266411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd7618-4e05-4e24-b104-95b049c6b6f2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.272644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 439/893] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/role_assignments?user.id=54bf77c17b884999a745b47787217a5a&scope.project.id=86229daf0943484393a30086a3139985&role.id=5c09a21ee6d942068400f80f686e0132 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:14.274358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f9819-f9bf-48f1-b659-6acb2f0158d9 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.274769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f9819-f9bf-48f1-b659-6acb2f0158d9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.275063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f9819-f9bf-48f1-b659-6acb2f0158d9 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.276573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 440/894] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:14.282215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.282500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.282780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.301966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:14.302780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.303322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.303948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.304418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.304877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.306744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.308447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.310103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.317701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 455/895] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:14.323033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.341883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.391101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 441/896] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:14.414885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.448042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.454245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae55b1-5283-4be9-90ae-5504726db503 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.502885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.504228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.504746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.505749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.506968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.507766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.510290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.522465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 456/897] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:14.524254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.531054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d15904-ce08-4e13-853e-273b08a329dc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.572401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.574028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.574542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.575022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.576451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.578540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:14.596573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:14.597626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:14.598468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:14.601651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 442/898] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:14.608194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.621588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.657342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502918-ab21-4221-be8a-3df7bcf9a2be None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.699286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.699873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.700159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.700438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.701225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] RBAC: Adding query filter params (interface=public, service_id=bdd4dca9d3914bbcab43cbb312ce06d0, region_id=RegionOne) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.701538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.702906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.713624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 457/899] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/endpoints?service_id=bdd4dca9d3914bbcab43cbb312ce06d0&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:14.714675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.728616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.736130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cfcc5-98c2-427a-9007-6dcffcf8df6f None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.775529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.776125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.776457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.776735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.777458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.777770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.779229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.784618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 443/900] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:14.792123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.853184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79eed-31a0-41c0-9ed0-ca1f1f9a6122 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.901356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.902312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.903031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.903354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.904365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.905743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abc240-1fe5-4ef0-9378-1554a7b253c6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.906429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463c021-9938-4299-bcd5-5a3737308a60 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.911835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 458/901] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:59:14.916155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de55b-03cb-43c8-8e12-8fb85af1bbf3 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.916155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de55b-03cb-43c8-8e12-8fb85af1bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.916155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de55b-03cb-43c8-8e12-8fb85af1bbf3 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.916155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 459/902] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:14.921314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.921610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.922010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.939192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:14.943904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:14.945945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.958265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.958265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.958265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.958265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.958664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:14.958664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.958664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.967337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 444/903] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:15.041694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 460/904] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:15.066748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:15.097461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:15.185807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e454-76d2-485d-a493-265416ec3624 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:15.222443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:15.223224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:15.223512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:15.223786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:15.224446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:15.225830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:15.238935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:15.239946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:15.240791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 445/905] 10.222.0.59 () {56 vars in 1132 bytes} [Thu Jul 14 10:59:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:15.247362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:15.354988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6595f7-85b2-4f3a-a3af-07420710d030 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:15.387163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:15.392108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:15.392708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:15.393035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:15.393318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:15.394029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:15.394605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:15.396005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:15.401872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 461/906] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:15.407502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:15.504851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:15.520987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fced1-9afc-4508-bd53-431e2c08f37a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:15.558697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:15.559429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:15.559720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:15.559996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:15.560702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:15.561021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:15.562401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:15.567787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 446/907] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:15.573701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:15.686801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556e192-1c98-4672-ae6e-848aca020137 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:15.723716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:15.724730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:15.725211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:15.725697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:15.726726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] RBAC: Adding query filter params (interface=public, service_id=7e4d23db91614a5bb6a451596ff32a72, region_id=RegionOne) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:15.727370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:15.729912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ad76-af1d-46d8-b59e-8ae50b41967a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:15.735760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 462/908] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:15 2022] GET /identity/v3/endpoints?service_id=7e4d23db91614a5bb6a451596ff32a72&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:15.747220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:15.854401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b336-5cba-4497-85d7-6546048146fb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:15.890027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:15.891081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:15.891585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:15.892096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:15.892987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:15.894545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5ea3-8765-4fcf-90c8-368f2fa23081 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:15.907543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 447/909] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:15 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:59:16.372565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169cf5a-e141-407f-bbe7-da0635bd0395 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:16.373063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169cf5a-e141-407f-bbe7-da0635bd0395 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:16.373168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169cf5a-e141-407f-bbe7-da0635bd0395 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:16.374748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 463/910] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:16.380397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:16.380695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:16.380980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:16.397808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:16.399280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:16.409573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:16.469252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 448/911] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:16.491940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169cf5a-e141-407f-bbe7-da0635bd0395 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:16.557479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:16.601140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169cf5a-e141-407f-bbe7-da0635bd0395 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:16.638870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:16.639720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:16.640008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:16.640287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:16.651964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:16.662737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:16.663702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:16.664529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 464/912] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:16.670337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:16.777248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247b901-52ad-41f5-bfd4-37a9048723ff None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:16.813176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:16.814085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:16.814511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:16.814925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:16.815861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:16.816310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:16.817815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:16.827445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 449/913] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:16.832812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:16.942945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000623-cbd9-4e53-884e-5ae38437e934 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:16.964790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:16.980107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:16.980919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:16.981206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:16.981485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:16.994793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:16.996293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-be42e371-5307-40b9-bd3b-f7b3afb9dd1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:17.014476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:17.015425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:17.016294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:17.016294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:17.016294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:17.016294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:17.016294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 465/914] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:17.021518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.031177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf0c-16c9-4686-a94e-0fa76697b2d9 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.031479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf0c-16c9-4686-a94e-0fa76697b2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.031758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf0c-16c9-4686-a94e-0fa76697b2d9 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.032781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.033378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 466/915] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:17.038293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.038687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.038850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.056727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:17.057941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:17.134581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61580ea-3e60-4e4e-8421-f071ecb9e491 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.139396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 467/916] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:17.157586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.178526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.180146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.180726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.181428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.182426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:17.182899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.184488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.201546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 450/917] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:17.210793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.282467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044f228-3cf3-46f0-95d5-6939556d83f6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.284164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.305164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.318497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.319413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.319813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.320156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.320933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.322933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-e81f2eb6-41b7-45d9-9078-36f6808cd6ac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:17.336101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:17.341711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:17.342718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 468/918] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:17.343977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5413-e416-45a5-a124-27395881da27 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.343977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5413-e416-45a5-a124-27395881da27 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.343977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5413-e416-45a5-a124-27395881da27 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.343977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 469/919] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:17.348220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5413-e416-45a5-a124-27395881da27 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.353746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705ef51-7338-4bb3-ab03-2839c83ecdf5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.389146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.389985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.390273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.390548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.398867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.401000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-790a18ad-d065-4132-9300-f6721a84c1c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:17.417710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:17.418932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:17.420105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:17.422256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 451/920] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:17.423597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.424090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.424562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.441594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:17.442790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:17.458578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5413-e416-45a5-a124-27395881da27 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.496127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.496796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.497080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.497360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.498068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:17.498377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.499775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.504778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 470/921] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:17.505636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.517410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 452/922] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:17.524572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.585339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.622168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455aa8-be46-4c9b-a64d-2d16ecb87447 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.633837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fad55-a8d2-4667-a9be-ad4be7036118 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.661255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.661966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.662299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.662576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.663550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:17.663863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.665905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.678746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.679765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.680243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.680757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.681944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:17.682446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.684092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.684531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 471/923] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:17.685408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.693685 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 453/924] 10.222.0.59 () {56 vars in 1114 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:17.694833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.785751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.801114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1f894-a16a-4782-be91-c6c09a7c2dba None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.804526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01617da3-a0ed-445e-ad2e-6e82b50ffdb0 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:17.825947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:17.837392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.842530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.842530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.842530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.842530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.842851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:17.842955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.843032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:17.843032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:17.843032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:17.854017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:17.856155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:17.874777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:17.875884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 454/925] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:17.876838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:17.924318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 472/926] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:17 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:17.925457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.016808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89535f6-5e94-4700-83c7-6d27deb82174 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.058678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b7bc7-cad9-4b09-9ce3-c1b0802d19f2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.069341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.070971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.072680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.073340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.074376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.076807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb1865-8cd2-45b0-bd96-9dbad1adb22f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.098972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.104021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f3df-1834-4789-8a76-ee5bf402ade2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.114408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 473/927] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:18.115913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043443e5-bbb0-417b-8583-85ad0cf8b992 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.116197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043443e5-bbb0-417b-8583-85ad0cf8b992 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.116473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043443e5-bbb0-417b-8583-85ad0cf8b992 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.117835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 474/928] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:18.121159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043443e5-bbb0-417b-8583-85ad0cf8b992 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.125994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.158580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 455/929] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:18.159899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.160209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.160470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.192224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:18.196546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:18.223825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.285061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 456/930] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:18.289533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043443e5-bbb0-417b-8583-85ad0cf8b992 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.310837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.336076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.337569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.337945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.338225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.346174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.356024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:18.362699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:18.363640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:18.364509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 475/931] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:18.369694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.440320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37bdce-3d60-42c2-bcc3-3ff8177d1825 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.447773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.486221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.487054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.487631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.487918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.488589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.490916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.500672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29642b-b6c1-44b3-96cc-edfa92ed909d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:18.509512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:18.510547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:59:18.511441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:18.512233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 457/932] 10.222.0.59 () {56 vars in 1132 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:18.518145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.541481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.542150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.542432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.542842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.543615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:18.543924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.544475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.546005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.558692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 476/933] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:18.564339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.635581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c347b-627c-4670-b3e6-245797841e9b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.672346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.672800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.673086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.673367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.674152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:18.674449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.675912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.676663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e482-7035-4d33-a0c6-00686c7ea1e3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.682662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 458/934] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:18.692363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.701035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.715917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.717031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.717856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.718086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/54bf77c17b884999a745b47787217a5a/roles/5c09a21ee6d942068400f80f686e0132` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.720626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=54bf77c17b884999a745b47787217a5a, role_id=5c09a21ee6d942068400f80f686e0132)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.722590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.783158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.799323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 477/935] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:18 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/54bf77c17b884999a745b47787217a5a/roles/5c09a21ee6d942068400f80f686e0132 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:18.805503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da23d5a-f1a7-47d4-b798-2df47c91566d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.845031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.845626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.845914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.846193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.846913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:18.847260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.848675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96897f9a-bc35-4fc7-8133-a67f064d8f52 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.855392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 459/936] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:18.860903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.988731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15169-2d57-454a-aef5-e5ca6197838e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:19.026839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:19.027557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:19.027839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:19.028116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:19.028862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:19.030232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c79e7-5d69-4614-aace-57cac4303674 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:19.118196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 478/937] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:20.079218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c4c36-4fc9-43e1-938f-0829ad695a07 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.079771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c4c36-4fc9-43e1-938f-0829ad695a07 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.079771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c4c36-4fc9-43e1-938f-0829ad695a07 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.081412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 460/938] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:20.086931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef06b7-d720-4798-9df0-80d8c72535f1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.087349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef06b7-d720-4798-9df0-80d8c72535f1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.087608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef06b7-d720-4798-9df0-80d8c72535f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.105563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e7ef06b7-d720-4798-9df0-80d8c72535f1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:20.106753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e7ef06b7-d720-4798-9df0-80d8c72535f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:20.136955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:20.187244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 479/939] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:20.206326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c4c36-4fc9-43e1-938f-0829ad695a07 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.315986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c4c36-4fc9-43e1-938f-0829ad695a07 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.332436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d410-767b-4ece-af83-f82d9b1312e4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.332709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d410-767b-4ece-af83-f82d9b1312e4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.332924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d410-767b-4ece-af83-f82d9b1312e4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.334557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 480/940] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:20.339577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.339870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.340155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.352028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.353515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.353801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.354076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.354732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.356180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.360414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:20.361860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:20.369045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:20.370121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:20.370971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:20.375311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 461/941] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:20.382244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.398909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:20.455172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 481/942] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:20.482065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.494567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf1a59-e154-4930-aa5a-076d82b69d61 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.532111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.532790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.533078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.533350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.534457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:20.534774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.536480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.544581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 462/943] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:20.549828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.599067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:20.611526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10677-d804-4e28-96e5-e63e72417c2b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.646497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:20.658263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.659008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.662962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.662962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.662962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.662962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.664434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING keystone.server.flask.common [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:59:20.676881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3e92-f07f-4abd-905f-7e54c7f38ad9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.716157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.sql.core [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:59:20.716157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:59:20.716157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [parameters: {'id': '637ed248a92d4c2b9c7957f205cd8b4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:59:20.716157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80484) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:59:20.716743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.717591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.718070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.718508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.719405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:20.719917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.721505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.732628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 463/944] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:20.734224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 637ed248a92d4c2b9c7957f205cd8b4b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 637ed248a92d4c2b9c7957f205cd8b4b. Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:59:20.736629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application result.read() Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 10:59:20.737621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application yield resource Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise self.value Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application session.commit() Jul 14 10:59:20.738700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:59:20.739616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:59:20.741092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application result.read() Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:59:20.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [parameters: {'id': '637ed248a92d4c2b9c7957f205cd8b4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.744516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:20.746143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 637ed248a92d4c2b9c7957f205cd8b4b. Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 482/945] 10.222.0.59 () {60 vars in 1140 bytes} [Thu Jul 14 10:59:20 2022] POST /identity/v3/projects => generated 262 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:20.748530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.874159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d218-e4ab-4b96-83f1-969896afabdc None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.893943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20ead-03d3-42a6-b1de-08ce5a5a7dca None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:20.912786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.914017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.914497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.915160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.922893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.924891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.936719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:20.937401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:20.937691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:20.937972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:20.938781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] RBAC: Adding query filter params (interface=public, service_id=044f9913af95480db6d27fd98cb67080, region_id=RegionOne) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:20.939110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:20.940512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:20.942267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:20.943365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:20.944264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:20.947248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 464/946] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/endpoints?service_id=044f9913af95480db6d27fd98cb67080&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:20.948624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 483/947] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:20.955481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.956304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:20.958906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:20.969173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.069711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b538-e5e7-419e-93a2-980c8e21cd12 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.075645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553d594-16e0-413c-ba09-dcc219ea7e29 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.105521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.106181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.106470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.106745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.107543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:21.107851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.109282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371522-5b98-439a-b93b-ef022cde6203 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.115990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.117254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.117831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.118307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.119674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.121386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a06bc-a85c-4f23-98e1-ab7a41a9dc0e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.135644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 484/948] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:21.135644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461a49-dfaf-41c7-ad2e-bcdf9655b7e4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.135644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461a49-dfaf-41c7-ad2e-bcdf9655b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.135644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee461a49-dfaf-41c7-ad2e-bcdf9655b7e4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.135644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 485/949] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:21.136176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 465/950] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:20 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:59:21.140452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.140750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.142963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.161128 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:21.162778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:21.242197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.248494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 486/951] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:21.276653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d0acb-bba7-4532-89ee-24674e359a1d None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.279256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d0acb-bba7-4532-89ee-24674e359a1d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.279386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.279487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d0acb-bba7-4532-89ee-24674e359a1d None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.279487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 466/952] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:21.284680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.285009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.285246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.306000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:21.307533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:21.389432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.399721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 467/953] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:21.415212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03d97a-9f1c-4c4b-b8f0-7f1117ef8560 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.424836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.452452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.453265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.453683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.453969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.466005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:21.474893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:21.475937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:21.476839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 487/954] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:21.484130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.538500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc7d0-dc2e-42e8-b671-ff20cbb3174e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.568285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.575361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.576232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.576520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.576799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.580719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.580719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:21.592151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:21.593555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:21.594800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:21.596197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 468/955] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:21.601237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.624581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391462b-d17a-4028-9576-af8c7ba18a68 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.669068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.670570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.673098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.673453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.673799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.674624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:21.675019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.676482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.686610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 488/956] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:21.691982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.725862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ee28-9e72-412c-9221-247b85726914 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.766187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.766869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.767150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.767445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.768149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:21.768505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.769820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.775779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 469/957] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:21.781110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.820511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780a45-0479-48be-9bc9-11eba800b471 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.843007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.858327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.859323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.859531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.859806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.872182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.873730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:21.893692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:21.894802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:21.895853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:21.895853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:21.895853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:21.895853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:21.900517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 489/958] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:21.902028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:21.912599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298303a-ef97-4cb3-8852-5536633c915e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:21.944416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:21.958277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:21.958961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:21.959289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:21.959953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:21.960677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:21.960993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:21.963041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:21.975153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 470/959] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:22.022235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba149524-a1c8-414c-b40c-14cb20304917 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:22.063618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:22.064331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:22.064620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:22.064897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:22.065631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:22.065939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:22.067364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:22.082179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 490/960] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:22.088589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:22.206861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373cf4-34e7-4acf-b111-b0e9d90d36b2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:22.241861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:22.242706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:22.243233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:22.243657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:22.251358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:22.252803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:22.270801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:22.271891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:22.273504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 471/961] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 10:59:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:22.280987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:22.315894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:22.391468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f1ea-9852-4c7a-9130-c99f208c4184 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:22.427468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:22.428530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:22.429016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:22.429490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:22.431062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:22.431656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:22.433400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:22.452494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 491/962] 10.222.0.59 () {56 vars in 1149 bytes} [Thu Jul 14 10:59:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:22.458291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:22.513789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:22.566551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00a9a-9a25-49a2-9a9a-d501957c7f10 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:22.601262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:22.602448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:22.602746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:22.603112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:22.674393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:22.675912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:22.725758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:22.779630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 472/963] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:22.788472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:22.879093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:22.889442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed871083-1b97-4f03-8db5-107374fec264 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:22.926176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:22.927435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:22.928069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:22.928855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:22.940218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:22.941752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:22.960501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:22.962328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:22.963431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:22.963431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:22.963431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:22.963431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:22.967156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 492/964] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:22.970082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.079524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629f109-c93e-478e-a298-c8015d39306a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.115826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.116681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.117081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.117495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.118331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:23.118748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.120337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78cff-a85b-418c-bebf-b01af4011941 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.132878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 473/965] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:23.138527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.250934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1be1f-de4b-4ed3-87ed-288885435803 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.287397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.288185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.288468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.288742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.295884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afc15e-fccb-492e-8ebd-62a24fb84917 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.296314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.296414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afc15e-fccb-492e-8ebd-62a24fb84917 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.296414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afc15e-fccb-492e-8ebd-62a24fb84917 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.297611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.298045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 474/966] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:23.302811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.303099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.303490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:23.312802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:23.313910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:23.314995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:23.316588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 493/967] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:23.322100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:23.323154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.324519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:23.327462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.328834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.409842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 475/968] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:23.429527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.435568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb95cd-f696-4c23-9a39-931079dfabf1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.475514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.476176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.476459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.476756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.478028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:23.478398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.480344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.499205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 494/969] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:23.503004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.548970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b25b3-1dec-43a8-9ff8-f83e6c5a0c42 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.572528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.586203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.587041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.587350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.587623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.588278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.590001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-3f2ebaf8-cce4-4638-8fb0-6cfb651ad523 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:23.602330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:23.603516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:23.604355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 476/970] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:23.605151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3273fd-e39a-4bc1-a4d1-b21312dc232d None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.605151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3273fd-e39a-4bc1-a4d1-b21312dc232d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.605274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3273fd-e39a-4bc1-a4d1-b21312dc232d None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.610478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 477/971] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:23.611267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3273fd-e39a-4bc1-a4d1-b21312dc232d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.614090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.616447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f403-0079-48ec-81a3-7a2ca2fffd00 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.653538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.654202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.654491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.654769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.655642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] RBAC: Adding query filter params (role.id=5c09a21ee6d942068400f80f686e0132, scope.project.id=86229daf0943484393a30086a3139985, user.id=54bf77c17b884999a745b47787217a5a) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:23.655954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.657394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4c6a-cd0e-47c7-b71d-203f9e01c753 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.664043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 495/972] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/role_assignments?user.id=54bf77c17b884999a745b47787217a5a&scope.project.id=86229daf0943484393a30086a3139985&role.id=5c09a21ee6d942068400f80f686e0132 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:23.665732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.666018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.666298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.686218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:23.688526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:23.744327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3273fd-e39a-4bc1-a4d1-b21312dc232d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.788160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 496/973] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:23.797018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.798148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.798659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.799210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.800198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:23.800728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.802453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.809663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 478/974] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:23.818642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.820085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:23.879724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.901211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:23.936495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c73daa-c8c2-4ef5-8d79-4d4d1d9573da None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.943183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d0277-9d7c-4698-934b-5e846fec049b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:23.977667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.979399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.980046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.980646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.981731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.983466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.984410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:23.984804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING keystone.server.flask.common [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:59:23.985082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.985371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.985646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.986359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:23.986664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:23.988237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:23.996627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 497/975] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:24.002297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:24.024711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.sql.core [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:59:24.024711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:59:24.024711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [parameters: {'user_id': 'adf7bed373e845c3bbe16217da844b3f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:59:24.024711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80483) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result.read() Jul 14 10:59:24.032168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:59:24.033057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application yield resource Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise self.value Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application session.commit() Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:59:24.034005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:59:24.034910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:59:24.035873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application result.read() Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:59:24.036824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [parameters: {'user_id': 'adf7bed373e845c3bbe16217da844b3f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:24.037732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 10:59:24.038518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 479/976] 10.222.0.59 () {60 vars in 1131 bytes} [Thu Jul 14 10:59:23 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:24.044056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:24.132131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b7ec1-86a7-40fb-aa49-283833cb065c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:24.155220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:24.159006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf716dd-88c5-49ab-8611-a0385d59176d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:24.172033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:24.172696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.172983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.173306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.174076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:24.175473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ba2b1-11a6-4920-894f-88eb4832e1ce None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:24.198807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:24.206325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:24.207139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.207433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.207709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.215940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:24.217569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-5b42b034-4542-46ff-afe4-5c464fbc94c9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:24.232904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:24.233902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:24.234924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 480/977] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:24.236275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044f45-7f2a-4698-b8b8-238da594bcd5 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.236275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044f45-7f2a-4698-b8b8-238da594bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.236275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f044f45-7f2a-4698-b8b8-238da594bcd5 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.240232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 481/978] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:24.241268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f044f45-7f2a-4698-b8b8-238da594bcd5 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:24.245370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 498/979] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:59:24.248991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.248991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.248991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.271081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:24.273633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:24.363957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 499/980] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:24.387392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:24.394555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f044f45-7f2a-4698-b8b8-238da594bcd5 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:24.442013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:24.442726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.442969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.443290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.444105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:24.444416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:24.446070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7744e-b11d-451e-a244-3edffdebbadb None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:24.494835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 482/981] 10.222.0.59 () {56 vars in 1101 bytes} [Thu Jul 14 10:59:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:24.515383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767d10-7109-482b-b55e-8e3cce9867f1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:24.517133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:24.556638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:24.557335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:24.557620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:24.557904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:24.558652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:24.560116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899ca4-0506-4005-96e1-8410b6acca25 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:24.615689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 500/982] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:24 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:25.863526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ee83a-46d5-491a-b137-31ec0f9bef40 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:25.864071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ee83a-46d5-491a-b137-31ec0f9bef40 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:25.864071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ee83a-46d5-491a-b137-31ec0f9bef40 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:25.865630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 483/983] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:25.871204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:25.871616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:25.872134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:25.889509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:25.890702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:25.966741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 501/984] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:25.989490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ee83a-46d5-491a-b137-31ec0f9bef40 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.041244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.098002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ee83a-46d5-491a-b137-31ec0f9bef40 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.132189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.133539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.133813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.134100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] PATH_INFO: `/v3/services/image` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.134752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.136165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:26.148008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:26.149100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:26.149936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:26.157227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 484/985] 10.222.0.59 () {56 vars in 1105 bytes} [Thu Jul 14 10:59:25 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:26.157379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.253926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.265952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d549d-8b80-4ad6-b419-42066837035b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.302100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.303213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.303694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.304186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.306397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:26.306916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.308609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2c688-6acd-4422-8071-7ebaee56e4a5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.313750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 502/986] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:26.319560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.354438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac73d-f32e-4ca4-b926-2cc22ccfd621 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.354755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac73d-f32e-4ca4-b926-2cc22ccfd621 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.355008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ac73d-f32e-4ca4-b926-2cc22ccfd621 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.356426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 503/987] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:26.361324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.361612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.361898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.379014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:26.380234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:26.436127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e52ee-5d48-46a2-b198-e48a054ef65e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.447405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.457219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 504/988] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:26.471343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.472016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.472301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.472577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.473282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] RBAC: Adding query filter params (type=image) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:26.473591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.474980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.481146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.481964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 485/989] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:26.576962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.597916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7e4de-e31a-442a-960a-1b89e57e3826 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.637993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.638874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.639189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.639469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.641421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.641528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9815349f-d24e-4ce7-8b35-87801d58dbf8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:26.655042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:26.656077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.656926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 505/990] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:26.662453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.670639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8df4-2b8e-4528-bc9a-040187a201dc None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.670927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8df4-2b8e-4528-bc9a-040187a201dc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.671239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8df4-2b8e-4528-bc9a-040187a201dc None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.672855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 506/991] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:26.677571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.677860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.678142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.694643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:26.696021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:26.768778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 507/992] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:26.778524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83959ae4-88b8-48bf-a079-82f011f96134 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.792516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.815243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.815911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.816195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.816466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.817165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:26.817470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.818827 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.825754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 486/993] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:26.831261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.874367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.875391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:26.899511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d5c9-f9b9-4b5e-a511-29e8e3a0777a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.934441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.935275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.935557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.935826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.936482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.937859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.946870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f180b5b-9c5f-4eed-94b5-e4d078e4fcd9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:26.951563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:26.952610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:26.956029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 508/994] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:26.961255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:26.986995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:26.987690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:26.987976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:26.988249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:26.988956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:26.989265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:26.990631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:26.997235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 487/995] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:27.002508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:27.082086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5399c-8269-4064-9b99-45d2eebc09dd None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.110159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5c234-857c-4dfe-93be-3b57d7b7b746 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.111920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:27.118267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.118963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.119588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.119588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.120248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:27.120585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.122010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.132501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 509/996] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:27.133865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:27.138553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:27.146440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.147140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.147676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.147962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.149365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] RBAC: Adding query filter params (interface=admin, service_id=044f9913af95480db6d27fd98cb67080, region_id=RegionOne) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:27.149680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.151326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.157146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 488/997] 10.222.0.59 () {56 vars in 1249 bytes} [Thu Jul 14 10:59:27 2022] GET /identity/v3/endpoints?service_id=044f9913af95480db6d27fd98cb67080&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:27.162505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:27.247680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7d28c-e472-4fad-9368-6e6fb90e3688 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.275366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e545-5288-4d04-b1d7-1dd8764274bc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.283717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.284446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.284704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.284985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.285694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:27.286006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.287402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.297480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 510/998] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:27.298494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:27.315649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.316322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.316609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.316891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.317577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.318955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.335156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 489/999] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:27 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:59:27.385797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:27.423424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e86cf-65ab-4d37-9566-aa25610901b9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.462313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.463440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.463997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.464496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.465622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] RBAC: Adding query filter params (interface=public, service_id=0c312040438141c08bdfd44c84f4d596, region_id=RegionOne) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:27.466157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.467863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154290b-9600-4201-93c8-826a5f1f3374 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.474448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 511/1000] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:27 2022] GET /identity/v3/endpoints?service_id=0c312040438141c08bdfd44c84f4d596&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:27.480784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:27.516638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:27.588581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c07d03-06b6-430c-ae66-fb52a281cab5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:27.625195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:27.625869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:27.626157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:27.626438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:27.627144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:27.628535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6aed9-b1f0-4e65-b4f3-c1347da42545 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:27.635105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 490/1001] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:27 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:59:28.549980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a752-6ce4-427a-9442-25f28feed277 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.550497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a752-6ce4-427a-9442-25f28feed277 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.550611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a752-6ce4-427a-9442-25f28feed277 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.552082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 512/1002] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:28.557258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dd00-198f-4a08-867f-2627a4c33d2f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.557647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dd00-198f-4a08-867f-2627a4c33d2f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.557811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3dd00-198f-4a08-867f-2627a4c33d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.575334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d8b3dd00-198f-4a08-867f-2627a4c33d2f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:28.576850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d8b3dd00-198f-4a08-867f-2627a4c33d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:28.646100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 491/1003] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:28.669160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a752-6ce4-427a-9442-25f28feed277 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:28.773766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a752-6ce4-427a-9442-25f28feed277 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:28.802886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:28.808692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:28.809363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.809649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.809926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.810656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:28.812062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3758e-f198-4cc1-ab68-ee471e3f4919 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:28.935159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 513/1004] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:28 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:29.397093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e679-1217-4dc5-bdae-8cb01d0e00eb None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.397777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e679-1217-4dc5-bdae-8cb01d0e00eb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.399775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e679-1217-4dc5-bdae-8cb01d0e00eb None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.400022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 492/1005] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:29.405235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c308-ad99-4459-9221-119b23d93b22 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.405527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c308-ad99-4459-9221-119b23d93b22 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.405803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5c308-ad99-4459-9221-119b23d93b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.450849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-18f5c308-ad99-4459-9221-119b23d93b22 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:29.451989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-18f5c308-ad99-4459-9221-119b23d93b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:29.526929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 514/1006] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:29 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:29.550183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e679-1217-4dc5-bdae-8cb01d0e00eb None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:29.576001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:29.654663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e679-1217-4dc5-bdae-8cb01d0e00eb None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:29.689483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:29.690286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.690572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.690847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.691538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:29.692911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:29.703360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f93f8-4cec-4743-944c-d0521dd44d08 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.703810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f93f8-4cec-4743-944c-d0521dd44d08 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.703893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f93f8-4cec-4743-944c-d0521dd44d08 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.705354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 515/1007] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:29.705535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:29.706481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:59:29.707344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:29.710024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.710318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.710630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.711404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 493/1008] 10.222.0.59 () {56 vars in 1126 bytes} [Thu Jul 14 10:59:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:29.712458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:29.728354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:29.729541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:29.811282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 516/1009] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:29 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:29.811679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:29.816616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505387e-0a57-459e-9a25-7d2208a1c04b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:29.827996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:29.853322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:29.853994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.854278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.854556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.855295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:29.855607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:29.856983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:29.866457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 494/1010] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:29.868611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:29.879795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:29.936017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2db8cd-785e-45fa-9e8a-01bb845c1663 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:29.973684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:29.974346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.974627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.974897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.975415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361885a-3e5a-410d-8ad9-92d2cfc6c4d3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:29.975597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:29.977935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:29.989437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:29.990439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:29.991312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 517/1011] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:29.997340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:30.017098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:30.017774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.018063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.018343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:30.019049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:30.019421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:30.020882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:30.026629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 495/1012] 10.222.0.59 () {56 vars in 1122 bytes} [Thu Jul 14 10:59:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:30.032467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:30.055143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:30.106126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0737-9655-4fd3-a217-037845722c8c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:30.122693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:30.142522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b42a1-ecea-471e-b0db-8f116676ae08 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:30.147808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:30.148613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.148904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.149539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:30.150596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:30.151391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:30.153112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:30.158541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 518/1013] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:30.164630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:30.188841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:30.189514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.189800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.190075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:30.190816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:30.192366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:30.258454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 496/1014] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:59:30.271364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef2cf-be4f-4ff4-9c6b-d37d1d2f06c6 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:30.318405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:30.319084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.319401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.319797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:30.321101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:30.321416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:30.322833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958e9d7-9744-496a-b87e-d70caec90f4d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:30.333410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 519/1015] 10.222.0.59 () {56 vars in 1112 bytes} [Thu Jul 14 10:59:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:30.334498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:30.451535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:30.475567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed0759-294c-4360-a38d-e8dee4eb7a26 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:30.510431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:30.511021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.511345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.511628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:30.512380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:30.513749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158281bc-e136-4c3c-8ddb-6fb278382857 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:30.619147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 497/1016] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:30.998588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5eea-3374-4141-a7e9-b2537815cb5e None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:30.999153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5eea-3374-4141-a7e9-b2537815cb5e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:30.999237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c5eea-3374-4141-a7e9-b2537815cb5e None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.000591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 520/1017] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:31.005767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.006072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.006356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.025209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:31.026426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:31.100596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 498/1018] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:31.124052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c5eea-3374-4141-a7e9-b2537815cb5e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.161042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:31.234369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c5eea-3374-4141-a7e9-b2537815cb5e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.269603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.270314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.270603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.270879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.271561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.273272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:31.284914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:31.285870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:31.286726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:31.291157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 521/1019] 10.222.0.59 () {56 vars in 1105 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:31.292417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.354309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:31.397288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3802b-8fc8-46b9-866f-3a4dfa522723 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.434747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.435460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.435746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.436022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.436729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] RBAC: Adding query filter params (name=image) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:31.437038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.438415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.447936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 499/1020] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:31.450576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.547071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:31.550391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a2043-f0a7-47da-95e3-c58e21befdf9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.583897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.584561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.584844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.585122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.586390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:31.586698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.588084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.593011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 522/1021] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:31.598815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.697670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e553e-5f5b-409d-8dd0-1d2e79680034 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.720479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:31.732267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.732931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.733215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.733490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.734287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] RBAC: Adding query filter params (interface=public, service_id=eda3af6bfe074abe9b42008156841cf1, region_id=RegionOne) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:31.734603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.736015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873a613-95ab-4cdb-9b8a-9f8a808245f6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.741208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 500/1022] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/endpoints?service_id=eda3af6bfe074abe9b42008156841cf1&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:31.746712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.850141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956701d-4db1-4153-9d37-869d643988e8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.886094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.886760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.887043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.887357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.888042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.889403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f24e2-4e34-477f-a3ca-768d16aa9d2b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.903409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 523/1023] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:59:32.350453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0fe4-f34c-42e6-8107-8f601b56ee5b None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.350747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0fe4-f34c-42e6-8107-8f601b56ee5b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.351072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0fe4-f34c-42e6-8107-8f601b56ee5b None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.352550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 501/1024] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:32.357826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.358265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.358432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.375420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:32.376637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:32.402318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:32.448839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 524/1025] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:32.472444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0fe4-f34c-42e6-8107-8f601b56ee5b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.576832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0fe4-f34c-42e6-8107-8f601b56ee5b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.611525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.612327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.612613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.612894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.614076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.615554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da0273-610b-48fc-996e-f1b3a4645c1b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.617950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 502/1026] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:32.623627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.722038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f792ed-b304-4427-aa10-d61b354005d2 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.755971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.756636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.756924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.757867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.758587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.760048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.768005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf721c6-264a-4995-b7e0-6f303c0e22e2 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.768471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf721c6-264a-4995-b7e0-6f303c0e22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.768579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf721c6-264a-4995-b7e0-6f303c0e22e2 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.770038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 503/1027] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:32.775339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.775628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.775907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.792511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:32.793688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:32.843072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:32.851109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 525/1028] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:32 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:32.863906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 504/1029] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:32.889674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.904145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.002127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38facd9a-0c74-4acb-a191-9966605f06ff None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.041600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.042423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.042707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.042983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.055676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:33.063750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:33.064715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:33.065579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 526/1030] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:33.071393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.173865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3687cb-ad53-4349-a41f-6af751124b43 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.210943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.211645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.211927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.212205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.212949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:33.213258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:33.214615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:33.228160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 505/1031] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:33.229393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.299620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.330804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3487c-72a5-45f9-9fd9-de8ebe6f9ca0 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.369146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.370055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.370346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.370618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.382064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:33.384757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:33.402998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:33.403974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:33.404885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:33.404885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:33.404885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:33.404885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:33.407453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 527/1032] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:33.409964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.459923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.517021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d296ef-bb7d-44f3-91bf-8636738efea8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.551476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.552066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.552349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.552652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.553338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:33.553646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:33.556161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:33.568380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 506/1033] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:33.574747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.677986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06865b-3b11-4fba-8a32-da48307113df None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.701380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.713624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.714412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.715644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.715644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.723364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:33.724824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:33.739813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:33.740746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:33.747245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 528/1034] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:33.747845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.851246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e0a9-9e91-4e05-b2cd-3cfeb592e0b9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:33.858835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.887674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:33.888357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.888646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.888925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.889764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:33.890079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:33.891552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6645f26-9ee3-4c3b-a1dd-50e0ce7fec6c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:33.909654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 507/1035] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:33.915791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.941910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7049ddc-c791-422c-9d31-439d0db62bd8 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.942213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7049ddc-c791-422c-9d31-439d0db62bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.942496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7049ddc-c791-422c-9d31-439d0db62bd8 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.944101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 508/1036] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:33.948985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.949272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.949552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.968541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:33.969745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:34.025150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1ae1-d0dd-45d6-889e-bb402fdd0749 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.046018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 509/1037] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:34.060820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.061477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.061761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.062035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.069713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.113525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.131549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.133515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.168468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.180411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec1a7a-c94e-48cb-9a0d-76035feba3b2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.220786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.221617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.221906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.222180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] PATH_INFO: `/v3/services/image` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.222831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.224234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:34.236372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:34.237409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:34.240255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 510/1038] 10.222.0.59 () {56 vars in 1105 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:34.243603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.248670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 529/1039] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:34.254356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.342678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.349299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53b71a-e1b2-4e0f-8c36-467c6624f513 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.362782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f94ec-8c7c-4735-a8da-72a97e94e2d1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.388805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.389403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.389685 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.389967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.390673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] RBAC: Adding query filter params (name=image) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:34.391033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.392443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.403138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 511/1040] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:34.404416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.405859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.406661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.406951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.407260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.418952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.420445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.430078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:34.439038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:34.443585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:34.445133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:34.445133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:34.445133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:34.445133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:34.445133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 530/1041] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:34.446399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.514353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe565e-72ed-4abf-a6c7-d1777669f580 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.549344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15c3ea-81c5-4fd0-9788-1a75e7892430 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.551720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.552399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.552681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.552961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.553688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:34.554731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.556147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.566005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 512/1042] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:34.566668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.585866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.586535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.586821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.587098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.587844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:34.588160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.589528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.591034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.601855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 531/1043] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:34.609115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.664447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.680259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b244-5ac1-4199-8ffe-0f610442a1f8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.712764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763ed3a-64df-4bed-bae0-df8f9996a730 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.718235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.718902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.719215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.719494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.720370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.721864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.746379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.747207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.747481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.747755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.754355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.755811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:34.770310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:34.771551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:34.773868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 532/1044] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:34.778203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.802812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 513/1045] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 10:59:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:34.843152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.894120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486a2b-89a9-4d06-a9ad-e0225ea2a447 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.938034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.938632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.938917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.939238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.940031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:34.940348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.941803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adf03e-5f33-402a-abe9-f5c336145908 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.959298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 533/1046] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:34.966184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:34.970747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f316b3b-97c8-415c-a885-2b1427fc3628 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.971039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f316b3b-97c8-415c-a885-2b1427fc3628 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.971345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f316b3b-97c8-415c-a885-2b1427fc3628 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.972693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 534/1047] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:34.978638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.978932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.979254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.995806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:34.997189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:35.068157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 535/1048] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:35.084128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103eed5-3eac-4e6e-b49b-7b3a36572d5d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:35.093917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:35.121623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:35.122277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:35.122557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:35.122828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:35.123702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] RBAC: Adding query filter params (role.id=5c09a21ee6d942068400f80f686e0132, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:35.124015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:35.125474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:35.131064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 514/1049] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:34 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=5c09a21ee6d942068400f80f686e0132 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:35.154326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:35.228721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fba87a-d504-40de-8b2b-6719bbc89939 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:35.267331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:35.268152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:35.268433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:35.268711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:35.269942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:35.271501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60145d3d-aaed-4bfb-be5a-2473325c60cf None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:35.274271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 536/1050] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:35.279926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:35.293198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:35.390333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421eb08-9b04-49ad-ab96-2f504479b7da None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:35.426088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:35.426767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:35.427055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:35.427356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:35.428072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:35.430049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1567dcc-28b0-4989-befb-2a282af56c6b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:35.493417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 515/1051] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:59:35 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:36.884424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a50a-af8c-48f9-b5ac-baad34b2306e None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:36.884946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a50a-af8c-48f9-b5ac-baad34b2306e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:36.885058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a50a-af8c-48f9-b5ac-baad34b2306e None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:36.886473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 537/1052] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:36.891681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fa8c-5896-4a96-a58d-2b2568695a26 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:36.892095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fa8c-5896-4a96-a58d-2b2568695a26 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:36.892310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fa8c-5896-4a96-a58d-2b2568695a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:36.909951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d976fa8c-5896-4a96-a58d-2b2568695a26 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:36.911169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d976fa8c-5896-4a96-a58d-2b2568695a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:36.969183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:36.979500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 516/1053] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:37.001674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994a50a-af8c-48f9-b5ac-baad34b2306e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.111204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994a50a-af8c-48f9-b5ac-baad34b2306e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.121712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.148196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.149335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.149987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.150727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.183796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2261b8-4827-4c6a-a68f-87fd8ad1211d None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.183796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2261b8-4827-4c6a-a68f-87fd8ad1211d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.183996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2261b8-4827-4c6a-a68f-87fd8ad1211d None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.185548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 517/1054] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:37.190552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.190885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.191107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.211255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:37.213406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:37.224426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.226823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.294665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 518/1055] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:59:37.317087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.339184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 538/1056] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:59:37.340301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.350837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.434684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2aa13-8bf3-40d3-8c9d-9a0a79deab67 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.448132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722a558-e422-4dd7-b8bd-b8322d80490c None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.461404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.472881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.474592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.474592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.474592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.485664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:37.486715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.487533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.487817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.488088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:37.494714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:37.495872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:37.496719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:37.501339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.502786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.504522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 519/1057] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:37.505369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:37.521177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:37.522598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:37.523563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:37.523563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:37.523563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:37.523563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:37.523563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 539/1058] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:37.529158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.592957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.612889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49ed93-579c-4aae-b739-925771ee6361 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.637122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d06de-c1f1-4a4a-813c-089fdc3534c7 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.648537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.649214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.649498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.649776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.650525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:37.650837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.652254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55ec8-7760-4fcb-a1ad-95e181f33d30 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.660260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 520/1059] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:37.661460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316a8f-9503-4c7a-b333-4f24000ef018 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.661749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316a8f-9503-4c7a-b333-4f24000ef018 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.662027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316a8f-9503-4c7a-b333-4f24000ef018 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.663857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 521/1060] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:37.666153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6316a8f-9503-4c7a-b333-4f24000ef018 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.675044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.676178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.676467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.676749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.677843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:37.678160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.680036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fe40a-009a-4ca5-a12f-e8ed1333547b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.691944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 540/1061] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:37.696654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.696965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.697248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.715377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:37.716522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.716638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:37.783486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6316a8f-9503-4c7a-b333-4f24000ef018 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.796381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 541/1062] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:37.822320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.823634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.824618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.824776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.825026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.839870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.841834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:37.860782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:37.861937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:37.863556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:37.863556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:37.863556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:37.863556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:37.863556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 522/1063] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:37.870057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:37.886716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.943411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafceec-0902-4600-91be-05c7f2e83a48 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.986710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.987548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.988352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.988635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.988910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.991862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d9dbf-b5b9-45f4-9da4-19b9afccae3a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.000887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:38.008953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:38.010013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:38.010899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:38.011747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 542/1064] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:38.016108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.027974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.028638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.028923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.029197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.029915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:38.030223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.031634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.043535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 523/1065] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.048988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.119889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931c4a-7f85-4f68-ab47-9ae8045a98a9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.129975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.148153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc293-580b-416b-baa9-acbb79c81562 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.154081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.154751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.155036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.155347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.156088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:38.156398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.157767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.167459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 543/1066] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.173166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.186169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.186968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.187300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.187575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.195822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.197217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:38.211565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:38.212528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:38.213859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 524/1067] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:38.218704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.236441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.278522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be362612-51f9-49be-b9ef-417e743c6e46 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.312956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.313812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.314091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.314360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] PATH_INFO: `/v3/users/99ad2667414a47309b9ff5156c935c0f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.318194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e068-6996-4d0b-ae7e-975753bf9647 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.327565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=99ad2667414a47309b9ff5156c935c0f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.328997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.330649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 544/1068] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/users/99ad2667414a47309b9ff5156c935c0f => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.336613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.356442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.357124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.357416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.357704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.358713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:38.359033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.360516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.366780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.378347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 525/1069] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.384699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.438428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af47b-2beb-47b3-8c79-da8976e58ac1 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.458062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.472466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.473202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.473435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.473708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.483447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcad7d8-e763-435c-b229-c724a0b0e84b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.516432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.517223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.517506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.517781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.529136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.530581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.539735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.541172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:38.547767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:38.548738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:38.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:38.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:38.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:38.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:38.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 526/1070] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:38.555425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.572166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.641511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 545/1071] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:59:38.647270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.657812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104755e-d90d-495b-85a8-90d8bf99d301 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.683337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.691420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.692080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.692360 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.692637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.693353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:38.693658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.695890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.707174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 527/1072] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.712708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.747433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26edd-1389-4385-8971-558ec9ed2cb5 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.775619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.780883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.781730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.782011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.782282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.784082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.785500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.788081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 546/1073] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.793562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.812562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9878c99-0f0d-4de3-ab34-b64786e01209 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.845919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.846713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.846992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.847310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.854458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.855927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:38.871021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:38.872101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 528/1074] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:38.878103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:38.892436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86feb38-b5ff-40d6-b63a-5c23f246823b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:38.924590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:38.927764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:38.928985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:38.929262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:38.929534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:38.930530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=c5a18a122b254160bca8429b45aa3448, user.id=99ad2667414a47309b9ff5156c935c0f) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:38.930894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:38.932368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:38.937954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 547/1075] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/role_assignments?user.id=99ad2667414a47309b9ff5156c935c0f&scope.project.id=c5a18a122b254160bca8429b45aa3448&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:38.985235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99e7e5-c9be-4814-93ea-e2cd5ec1816c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:39.022108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:39.022861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:39.023074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:39.023383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:39.024166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:39.024484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:39.025977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea04eec-bbb3-4056-8957-c52e15f05b94 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:39.039925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 529/1076] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:39.046008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:39.109399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:39.148418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c0d2-cb49-4397-bb37-86b160b01f03 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:39.185822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:39.186711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:39.186989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:39.187289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/5c09a21ee6d942068400f80f686e0132` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:39.189443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=c101547390374884b75d95b637267e79, role_id=5c09a21ee6d942068400f80f686e0132)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:39.190902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b1923-21de-4038-baa9-ff64d3a5d2c0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:39.252190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 548/1077] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:39 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/c101547390374884b75d95b637267e79/roles/5c09a21ee6d942068400f80f686e0132 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:59:40.965735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe168-2fa7-4ef6-84dd-f10d6c51f081 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.966291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe168-2fa7-4ef6-84dd-f10d6c51f081 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.966291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fe168-2fa7-4ef6-84dd-f10d6c51f081 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.967764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 530/1078] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:40.973152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.973569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.973743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.990982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:40.992296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:41.058061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 549/1079] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:41.080840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fe168-2fa7-4ef6-84dd-f10d6c51f081 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.182842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fe168-2fa7-4ef6-84dd-f10d6c51f081 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.192018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.226300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.227029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.227626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.228106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.238525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7a82d1f1-7de7-4db6-84fd-7d1b2bed094a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:41.246720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:41.247876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:41.253674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 531/1080] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:41.254371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.272969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b87127-31f7-44ef-98f3-98cae39df459 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.273254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b87127-31f7-44ef-98f3-98cae39df459 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.273536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b87127-31f7-44ef-98f3-98cae39df459 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.274884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 532/1081] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:41.279799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.280084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.280359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.297994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:41.299208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:41.307427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.365030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11496ee2-04f4-4417-85fb-b0d3e19b1a16 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.368487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 533/1082] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:41.392489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.399933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.400608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.400894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.401172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.401925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:41.402251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.403636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.412369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 550/1083] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:41.418484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.487011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.493043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ed40-8b95-466d-99ea-bc8b067db09a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.517063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceeba67-d0d9-4540-afd9-2596af3487a9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.527602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.528426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.528700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.528977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.531880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.539507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:41.547616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:41.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:59:41.549446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:41.551261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 534/1084] 10.222.0.59 () {56 vars in 1120 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:41.552431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.553936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.554312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.554587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] PATH_INFO: `/v3/users/99ad2667414a47309b9ff5156c935c0f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.555352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.555890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] RBAC: Authorizing `identity:get_user(user_id=99ad2667414a47309b9ff5156c935c0f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.557336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.559387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 551/1085] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/users/99ad2667414a47309b9ff5156c935c0f => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:41.565343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.731825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae093b1-bd84-44b1-85bc-405ab14e103a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.735905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd611c2-2934-42d3-90f1-f38b50cdf3f9 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.764984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.765582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.765870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.766151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.766902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:41.767244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.768663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.768950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.769812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.770098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.770374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.772487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.774156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.775933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 552/1086] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:41.782074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 535/1087] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:41.782205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.783021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:41.802750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.844770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.892349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32fc55-72ed-4c82-8deb-97bcbc65c3bf None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.897326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e20f4e-699e-48b9-91a9-7079eb8900b2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:41.928444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.929256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.929549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.930020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.933636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:41.934515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.934795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.935069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448/users/99ad2667414a47309b9ff5156c935c0f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.937036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c5a18a122b254160bca8429b45aa3448, user_id=99ad2667414a47309b9ff5156c935c0f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.938461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.945592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:41.947048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:41.965830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:41.967385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:41.968484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:41.968484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:41.968484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:41.968484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:41.968484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 536/1088] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:41.973321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.001167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 553/1089] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:41 2022] PUT /identity/v3/projects/c5a18a122b254160bca8429b45aa3448/users/99ad2667414a47309b9ff5156c935c0f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:59:42.059412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.095021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2474737-bd17-4e41-b653-85803a320c1e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.133131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.133825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.134112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.134391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.135157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:42.135475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:42.136913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:42.152952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 537/1090] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:42.158881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.277940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0273-0021-45d4-97b8-4ddc93c011ce None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.290443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.316901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.317624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.317904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.318181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.329728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:42.331625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:59:42.349662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:42.353661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:42.354689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 554/1091] 10.222.0.59 () {56 vars in 1093 bytes} [Thu Jul 14 10:59:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:42.357421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.470811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff920-d929-42e2-9f14-d46a2601f4be None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.502230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.508656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.510280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.510563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.510836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.513370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:42.513370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:42.513515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:42.534592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 538/1092] 10.222.0.59 () {56 vars in 1133 bytes} [Thu Jul 14 10:59:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:42.542106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.653193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005575a3-803f-44a8-a713-415cdc152413 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.687209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.687861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.688141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.688417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.727997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.756414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:42.757865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:42.853508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 555/1093] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:59:42.859180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.959321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6071-c489-4f58-8a4b-8160f590f247 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.993897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.994698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.994984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.995301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.005445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.007551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:59:43.025598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:43.026502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:43.027584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:59:43.027584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:59:43.027584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:59:43.027584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:43.027584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 539/1094] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:43.032395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:43.099359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:43.130561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb2f7-9cdf-48fc-96c5-7b2d9b4f6d1a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:43.165972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:43.166624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:43.166905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:43.167213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.167924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:43.168230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.169584 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.187484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 556/1095] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 10:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:43.188563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:43.229984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:43.287415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febb740-0e10-44da-8fe0-5554d513813e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:43.321012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:43.321796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:43.322094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:43.322351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.328829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.330261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:59:43.343905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:43.345091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 10:59:43.346031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 540/1096] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 10:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:59:43.351203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:43.450719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e4eda-e377-46a2-8b4c-19bd78a9b2b3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:43.479077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:43.484972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:43.485576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:43.485862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:43.486144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.486985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:43.487333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.492869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16644726-3a0d-469c-96e6-5a7fdf1299cc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.501323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 557/1097] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 10:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:43.507484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:43.606682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:43.609721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785ea1c-05a2-49d1-9d98-90a39c762b02 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:43.643830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:43.644490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:43.644777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:43.645057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.645891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] RBAC: Adding query filter params (role.id=5c09a21ee6d942068400f80f686e0132, scope.project.id=86229daf0943484393a30086a3139985, user.id=c101547390374884b75d95b637267e79) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:43.646517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.648016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c12d59-469b-45f3-9198-08890b9f8b97 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.654071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 541/1098] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:43 2022] GET /identity/v3/role_assignments?user.id=c101547390374884b75d95b637267e79&scope.project.id=86229daf0943484393a30086a3139985&role.id=5c09a21ee6d942068400f80f686e0132 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:44.088287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcde8e-dc76-437c-ab9d-92de22c90272 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.088804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcde8e-dc76-437c-ab9d-92de22c90272 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.088922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcde8e-dc76-437c-ab9d-92de22c90272 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.090228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 558/1099] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:44.095466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.095889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.096017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.114444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:44.115964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:44.185964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 542/1100] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:44.211227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcde8e-dc76-437c-ab9d-92de22c90272 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:44.313579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcde8e-dc76-437c-ab9d-92de22c90272 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:44.350329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:44.350329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.350579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.350772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.361223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:59:44.369450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:59:44.370453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 10:59:44.371485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 559/1101] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:59:44.376739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:44.474063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:44.481296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9e21-5d46-4ecd-9dd2-8635a2903933 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:44.515945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:44.516633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.516918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.517193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.518227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:44.518535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:44.520056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:44.528642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 543/1102] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:59:44.534453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:44.584720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:44.634046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ac7dc-3b5e-422d-b48b-5be7df586747 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:44.667656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:44.668512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.668800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.669083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] PATH_INFO: `/v3/users/99ad2667414a47309b9ff5156c935c0f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.670258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] RBAC: Authorizing `identity:get_user(user_id=99ad2667414a47309b9ff5156c935c0f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:44.671735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:44.673358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 560/1103] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/users/99ad2667414a47309b9ff5156c935c0f => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:44.679236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:44.775873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f412e1-e1d4-4908-b8f3-29a6d22a682e None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:44.808240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:44.808876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.809161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.809441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.832607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:44.874614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:44.876092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:44.970552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 544/1104] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:44.976343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:45.073342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8de52-6821-402a-9045-42ba8e5bdd03 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:45.089842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:45.105790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:45.106627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:45.106908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:45.107211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:45.115758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:45.115758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c9a8-e215-4a46-83ae-9e835d6b9a0c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:45.115758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 561/1105] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:45.117915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:45.172640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:45.217296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b771e4-367f-4006-ab7c-85f5f80baac7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:45.249139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:45.249786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:45.250068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:45.250344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:45.251182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=c5a18a122b254160bca8429b45aa3448, user.id=99ad2667414a47309b9ff5156c935c0f) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:45.251497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:45.252919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678bfd9-7fef-4880-a3ba-577d91aa8728 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:45.259059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 545/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:45 2022] GET /identity/v3/role_assignments?user.id=99ad2667414a47309b9ff5156c935c0f&scope.project.id=c5a18a122b254160bca8429b45aa3448&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:47.335139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaf8b4-3b61-4794-8ded-0b007ab03024 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:47.335682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaf8b4-3b61-4794-8ded-0b007ab03024 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:47.335682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaf8b4-3b61-4794-8ded-0b007ab03024 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:47.337059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 562/1107] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:47.342281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:47.342684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:47.342846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:47.360014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:47.361208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:47.429501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 546/1108] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:47.454631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaf8b4-3b61-4794-8ded-0b007ab03024 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:47.555599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaf8b4-3b61-4794-8ded-0b007ab03024 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:47.588360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:47.589134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:47.589417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:47.589689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:47.590834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:47.592352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f493127-1cca-43be-ab4e-2151abc22fde None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:47.593937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 563/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:47.599718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:47.650686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:47.699095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf3df8-9e82-41a5-8704-874734247652 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:47.732436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:47.733097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:47.733383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:47.733660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:47.734353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:47.735818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0c185-1ca0-42d1-a87d-1e13eae09d08 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:47.804949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 547/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 10:59:47 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:49.896588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc932-6c31-4542-9005-afb098b55560 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:49.903385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc932-6c31-4542-9005-afb098b55560 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:49.903385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdc932-6c31-4542-9005-afb098b55560 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:49.903385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 564/1111] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:49.904247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:49.904629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:49.904804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:49.921961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:49.923157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:49.988669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 548/1112] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:50.011782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdc932-6c31-4542-9005-afb098b55560 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:50.068191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:50.113027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdc932-6c31-4542-9005-afb098b55560 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:50.144704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:50.146350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:50.147174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:50.147629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] PATH_INFO: `/v3/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:50.149132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] RBAC: Authorizing `identity:get_role(role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:50.150496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:50.151899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 565/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:50 2022] GET /identity/v3/roles/d5708cbdc7894157a71389bd8517df5f => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:50.157613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:50.192065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:50.256281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e9ee3-3184-4dc0-9733-612dcb625ecc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:50.289451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:50.290316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:50.290599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:50.290873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] PATH_INFO: `/v3/users/f61d530edd3e427092bfaa71e116e35a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:50.303748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] RBAC: Authorizing `identity:get_user(user_id=f61d530edd3e427092bfaa71e116e35a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:50.305183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:50.306835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 549/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:50 2022] GET /identity/v3/users/f61d530edd3e427092bfaa71e116e35a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:50.312561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:50.414345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdccb1-e6a0-4431-bcdf-bf47433fc86f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:50.435909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:50.447501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:50.448172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:50.448458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:50.448735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:50.514175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:50.515650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:50.610480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 566/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:50.617262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:50.708344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:50.714861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a638be-8a49-4f75-b316-0f33a283651b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:50.748476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:50.749338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:50.749628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:50.749908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:50.751078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:50.752539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3da64e-a323-449f-bcc2-8e183cd47544 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:50.754235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 550/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:50 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:50.760908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:50.783947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:50.856886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41ad0b-97e6-4dc9-b9e3-c00a358a0c6d None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:50.890916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:50.891588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:50.891870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:50.892140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:50.892958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=c5a18a122b254160bca8429b45aa3448, user.id=f61d530edd3e427092bfaa71e116e35a) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:50.893267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:50.894674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a9338-e04f-4578-8dfb-b062eab2181e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:50.900763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 567/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:50 2022] GET /identity/v3/role_assignments?user.id=f61d530edd3e427092bfaa71e116e35a&scope.project.id=c5a18a122b254160bca8429b45aa3448&role.id=d5708cbdc7894157a71389bd8517df5f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:52.893189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ffcef-6dd4-456d-920d-395eaf52c0ec None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:52.893744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ffcef-6dd4-456d-920d-395eaf52c0ec None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:52.893744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ffcef-6dd4-456d-920d-395eaf52c0ec None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:52.895391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 551/1118] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:52.901199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:52.901634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:52.901765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:52.919380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:52.920578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:52.985233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 568/1119] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:53.008146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ffcef-6dd4-456d-920d-395eaf52c0ec None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:53.107830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ffcef-6dd4-456d-920d-395eaf52c0ec None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:53.140840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:53.141687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:53.141974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:53.142252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] PATH_INFO: `/v3/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:53.143799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] RBAC: Authorizing `identity:get_role(role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:53.145399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:53.147424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 552/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:53 2022] GET /identity/v3/roles/d5708cbdc7894157a71389bd8517df5f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:53.153252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:53.170514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:53.250277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0707c-1974-44ed-b566-eb18daaa5244 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:53.284136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:53.284996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:53.285275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:53.285546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] PATH_INFO: `/v3/users/f61d530edd3e427092bfaa71e116e35a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:53.286906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] RBAC: Authorizing `identity:get_user(user_id=f61d530edd3e427092bfaa71e116e35a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:53.288362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:53.289973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 569/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:53 2022] GET /identity/v3/users/f61d530edd3e427092bfaa71e116e35a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:53.295814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:53.325703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:53.392573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42922b61-73de-40fb-b47b-3f42a8c88ad2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:53.425821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:53.426615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:53.426998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:53.427449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:53.428625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:53.430056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed8949-7402-49f1-ac2f-28d2689466f0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:53.431758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 553/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:53 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:53.437554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:53.519343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:53.535344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8ea39-4660-4b58-86a4-0e87fb21eaa3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:53.568933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:53.569805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:53.570088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:53.570364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448/users/f61d530edd3e427092bfaa71e116e35a/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:53.572560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5a18a122b254160bca8429b45aa3448, user_id=f61d530edd3e427092bfaa71e116e35a, role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:53.573986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afd69f-4ef9-4df5-bc33-bb2f60ebc5f9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:53.626911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 570/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 10:59:53 2022] PUT /identity/v3/projects/c5a18a122b254160bca8429b45aa3448/users/f61d530edd3e427092bfaa71e116e35a/roles/d5708cbdc7894157a71389bd8517df5f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:55.617755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133063-1fe9-45d3-bbc9-c78822ef2854 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:55.618302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133063-1fe9-45d3-bbc9-c78822ef2854 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:55.618302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71133063-1fe9-45d3-bbc9-c78822ef2854 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:55.619713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 554/1124] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:55.624945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:55.625319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:55.625526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:55.642046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:55.643240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:55.706712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 571/1125] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:55.731004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133063-1fe9-45d3-bbc9-c78822ef2854 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:55.826086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:55.830632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71133063-1fe9-45d3-bbc9-c78822ef2854 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:55.864116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:55.865051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:55.865338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:55.865618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] PATH_INFO: `/v3/roles/d5708cbdc7894157a71389bd8517df5f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:55.867092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] RBAC: Authorizing `identity:get_role(role_id=d5708cbdc7894157a71389bd8517df5f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:55.868538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:55.869839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 555/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:55 2022] GET /identity/v3/roles/d5708cbdc7894157a71389bd8517df5f => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:55.875566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:55.950408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:55.975936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7eb30f-160a-40c4-8e12-dbbb76e202d0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:56.009074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:56.009935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:56.010218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:56.010492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] PATH_INFO: `/v3/users/f61d530edd3e427092bfaa71e116e35a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:56.011688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f61d530edd3e427092bfaa71e116e35a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:56.013118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:56.014747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 572/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 10:59:55 2022] GET /identity/v3/users/f61d530edd3e427092bfaa71e116e35a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:56.021394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:56.118149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9313-10f4-47ea-9742-744f4926b8bd None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:56.151677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:56.152253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:56.152536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:56.152813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:56.185543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:56.220005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:56.221459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:56.314832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 556/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 10:59:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:59:56.321557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:56.417397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02062a-f62c-4c59-baff-253ae76ae1a8 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:56.450776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:56.451645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:56.451925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:56.452195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] PATH_INFO: `/v3/projects/c5a18a122b254160bca8429b45aa3448` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:56.453336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] RBAC: Authorizing `identity:get_project(project_id=c5a18a122b254160bca8429b45aa3448)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:56.454752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97107dd-350f-4207-a2d3-f9a406c125e0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:56.456439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 573/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 10:59:56 2022] GET /identity/v3/projects/c5a18a122b254160bca8429b45aa3448 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:56.463140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:56.526786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:56.560469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc45cf8-6b6c-4274-8d11-59450e629464 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:56.592688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:56.593338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:56.593623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:56.593901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:56.594735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] RBAC: Adding query filter params (role.id=d5708cbdc7894157a71389bd8517df5f, scope.project.id=c5a18a122b254160bca8429b45aa3448, user.id=f61d530edd3e427092bfaa71e116e35a) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:56.595052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:56.596515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57335bc-3b82-4378-a3a4-d7d2d3d19d62 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:56.602255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 557/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 10:59:56 2022] GET /identity/v3/role_assignments?user.id=f61d530edd3e427092bfaa71e116e35a&scope.project.id=c5a18a122b254160bca8429b45aa3448&role.id=d5708cbdc7894157a71389bd8517df5f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:58.630048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57533f1-8c67-4d24-bf42-64f8fcd5ce04 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.630603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57533f1-8c67-4d24-bf42-64f8fcd5ce04 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.630739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57533f1-8c67-4d24-bf42-64f8fcd5ce04 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.632156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 574/1131] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 10:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:58.637347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.637748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.637935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.656274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:58.657728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:58.723473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 558/1132] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:59:58.746420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57533f1-8c67-4d24-bf42-64f8fcd5ce04 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:58.758980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:58.844999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57533f1-8c67-4d24-bf42-64f8fcd5ce04 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:58.877442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:58.878213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.878495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.878769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.880882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:58.882257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc6397-e825-45ec-b455-39db11434101 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:58.883865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 575/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 10:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:58.889806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:58.949439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:58.991889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb15a-0d23-44ef-827c-b9fd585d0a90 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:59.024217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:59.024864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:59.025146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:59.025419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:59.026103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:59.027561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde330a-4bd3-4f3a-ba02-a9440a79c256 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:59.107818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 559/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 10:59:58 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:01.170972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1802d7-63a5-45fb-b9aa-06668a371e44 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:01.171539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1802d7-63a5-45fb-b9aa-06668a371e44 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:01.171539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1802d7-63a5-45fb-b9aa-06668a371e44 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:01.172906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 576/1135] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:01.179099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:01.179099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:01.179566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:01.196637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:01.197814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:01.263080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 560/1136] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:01.288133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1802d7-63a5-45fb-b9aa-06668a371e44 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:01.365641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:01.388041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1802d7-63a5-45fb-b9aa-06668a371e44 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:01.421547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:01.422256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:01.422539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:01.422816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:01.424003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:01.425433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2d428-aa8e-43ae-adce-788ef7d8ff31 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:01.427633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 577/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:01.433414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:01.477936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:01.534175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda81b5-eb56-4a5f-9560-44829311c96b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:01.569044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:01.569709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:01.569996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:01.570275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:01.570975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:01.572440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c17c1-087c-4bc3-85c1-5d62a73dc77f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:01.672458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 561/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 11:00:01 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:03.743801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b40fc8-8d15-491e-8d5a-5b27f0c35f5c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:03.744341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b40fc8-8d15-491e-8d5a-5b27f0c35f5c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:03.744341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b40fc8-8d15-491e-8d5a-5b27f0c35f5c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:03.745704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 578/1139] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:03.750883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:03.751350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:03.751478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:03.770854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:03.772076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:03.837017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 562/1140] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:03.861666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b40fc8-8d15-491e-8d5a-5b27f0c35f5c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:03.964893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b40fc8-8d15-491e-8d5a-5b27f0c35f5c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:03.997399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:03.998140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:03.998405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:03.998677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.009068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:04.016554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:04.017707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:04.018775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 579/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:04.024742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.051961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.123510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9b2f-3444-419b-b682-a67ece185117 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.173574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.174258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.175620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.176089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.177098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:04.177513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.178903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.188327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 563/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:04.195037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.206478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.300368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b40b-368c-4102-a087-4962f09be79f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.333235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.334093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.334376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.334650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] PATH_INFO: `/v3/users/40d07f8e80e447d5a2e91f033642008a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.348325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] RBAC: Authorizing `identity:get_user(user_id=40d07f8e80e447d5a2e91f033642008a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.355070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.355070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 580/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:04 2022] GET /identity/v3/users/40d07f8e80e447d5a2e91f033642008a => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:04.357226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.446308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.455608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9512-0ffb-4ab1-8837-c98e558b8df8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.489619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.490248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.490531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.490807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.556396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.557781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.643781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.654123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 564/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:00:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:00:04.660685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.721104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.758952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd97e-28fe-4fcd-861c-24ef0a98d2c3 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.791564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.792420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.792705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.792985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] PATH_INFO: `/v3/projects/0757c82c01f442488a0d05018f546cbb` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.794155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] RBAC: Authorizing `identity:get_project(project_id=0757c82c01f442488a0d05018f546cbb)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.795609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40894fb-9dad-49da-b65e-b8c97c60c055 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.797305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 581/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:04 2022] GET /identity/v3/projects/0757c82c01f442488a0d05018f546cbb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:04.803931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.901107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e8e3-a259-48a3-80ad-1134893ebc5b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.934057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.934698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.934978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.935278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.936097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=0757c82c01f442488a0d05018f546cbb, user.id=40d07f8e80e447d5a2e91f033642008a) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:04.936408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.937832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1efde-c89a-40bd-8430-ffce584b463d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.943545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 565/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:00:04 2022] GET /identity/v3/role_assignments?user.id=40d07f8e80e447d5a2e91f033642008a&scope.project.id=0757c82c01f442488a0d05018f546cbb&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:06.946767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6e35-0eea-4a4e-b7ff-e83de764357d None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.947371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6e35-0eea-4a4e-b7ff-e83de764357d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.947479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6e35-0eea-4a4e-b7ff-e83de764357d None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.949070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 582/1147] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:06.954612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.955036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.955183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.971731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:06.973704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:07.021992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:07.039176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 566/1148] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:07.062834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa6e35-0eea-4a4e-b7ff-e83de764357d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:07.163108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa6e35-0eea-4a4e-b7ff-e83de764357d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:07.191592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:07.196301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:07.203480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:07.203480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:07.203480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:07.208353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:07.216732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:07.217832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:07.219042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 583/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:07.224682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:07.321161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c98af-3af1-45a8-bd9d-4abd2aacd76e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:07.355306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:07.355959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:07.356247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:07.356526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:07.357276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:07.357588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:07.358967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:07.366994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 567/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:07.372681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:07.470768 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79376a9-262c-4df3-bbd9-1add83493c81 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:07.502155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:07.503047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:07.503923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:07.504194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] PATH_INFO: `/v3/users/40d07f8e80e447d5a2e91f033642008a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:07.505359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] RBAC: Authorizing `identity:get_user(user_id=40d07f8e80e447d5a2e91f033642008a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:07.506785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:07.508429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 584/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:07 2022] GET /identity/v3/users/40d07f8e80e447d5a2e91f033642008a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:07.514148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:07.559896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:07.611264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b44985-28d6-4b1f-abef-10ee8630e659 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:07.643662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:07.645467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:07.645773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:07.646084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] PATH_INFO: `/v3/projects/0757c82c01f442488a0d05018f546cbb` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:07.647286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] RBAC: Authorizing `identity:get_project(project_id=0757c82c01f442488a0d05018f546cbb)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:07.648714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a67624-13e8-42b6-886f-ccd06ed7fd2c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:07.650417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 568/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:07 2022] GET /identity/v3/projects/0757c82c01f442488a0d05018f546cbb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:07.656252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:07.714036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:07.753950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e35e43-b2c7-434c-bac6-dc410df4fe53 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:07.786343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:07.787237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:07.787523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:07.787800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] PATH_INFO: `/v3/projects/0757c82c01f442488a0d05018f546cbb/users/40d07f8e80e447d5a2e91f033642008a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:07.789733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] RBAC: Authorizing `identity:create_grant(project_id=0757c82c01f442488a0d05018f546cbb, user_id=40d07f8e80e447d5a2e91f033642008a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:07.791182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c0352-fb30-4734-9417-cc07be585f7f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:07.859800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 585/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 11:00:07 2022] PUT /identity/v3/projects/0757c82c01f442488a0d05018f546cbb/users/40d07f8e80e447d5a2e91f033642008a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:09.856987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc7c3c-23c9-4918-8de2-ff779bc14b50 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.857515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc7c3c-23c9-4918-8de2-ff779bc14b50 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.857625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc7c3c-23c9-4918-8de2-ff779bc14b50 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.858936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 569/1154] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:09.864772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.865199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.865337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.881860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:09.883040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.948055 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 586/1155] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:09.972294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc7c3c-23c9-4918-8de2-ff779bc14b50 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.071991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc7c3c-23c9-4918-8de2-ff779bc14b50 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.086164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.105646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.106431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.108019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.108019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.116872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:10.124498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:10.125581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:00:10.126582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 570/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:10.131809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.182126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.232401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c423f-9a4d-425b-a66e-4613bfcacd1b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.264925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.265578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.265858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.266129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.266861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:10.267191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.269328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.277534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 587/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:10.283465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.380879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe798a3-c493-4693-9974-adf22c687495 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.414287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.415163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.415457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.415735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] PATH_INFO: `/v3/users/40d07f8e80e447d5a2e91f033642008a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.416912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] RBAC: Authorizing `identity:get_user(user_id=40d07f8e80e447d5a2e91f033642008a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.418350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.420039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 571/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/users/40d07f8e80e447d5a2e91f033642008a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:10.426347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.525714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de4770-735c-4db4-867c-681141ae7438 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.538577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.558895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.559558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.559840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.560110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.625919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.627386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.723678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 588/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:00:10.729400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.772540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.827957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90de9-eb1d-4b23-93bb-fd7e9bed9677 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.860263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.861109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.861395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.861696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] PATH_INFO: `/v3/projects/0757c82c01f442488a0d05018f546cbb` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.863489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] RBAC: Authorizing `identity:get_project(project_id=0757c82c01f442488a0d05018f546cbb)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.864930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d24383-28ee-4826-bf75-8c591f842899 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.866614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 572/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/projects/0757c82c01f442488a0d05018f546cbb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:10.872730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.888476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.974949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969031f-f379-4082-a68c-778a9e493857 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:11.009566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:11.010230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:11.010515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:11.010794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:11.011656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=0757c82c01f442488a0d05018f546cbb, user.id=40d07f8e80e447d5a2e91f033642008a) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:11.011970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:11.013393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d7364-ba16-4d95-b6a8-4f9a4cf47c9c None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:11.019512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 589/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/role_assignments?user.id=40d07f8e80e447d5a2e91f033642008a&scope.project.id=0757c82c01f442488a0d05018f546cbb&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:13.082709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45ce8-1a0f-43ab-b8d0-d802a3fd338a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:13.083283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45ce8-1a0f-43ab-b8d0-d802a3fd338a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:13.083283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45ce8-1a0f-43ab-b8d0-d802a3fd338a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:13.084664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 573/1162] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:13.089683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:13.090124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:13.090226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:13.106998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:13.108212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:13.172694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 590/1163] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:13.194724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45ce8-1a0f-43ab-b8d0-d802a3fd338a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:13.294546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45ce8-1a0f-43ab-b8d0-d802a3fd338a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:13.305735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:13.327894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:13.328533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:13.328821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:13.329101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:13.395494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:13.396892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:13.491215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 574/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:00:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:00:13.497169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:13.506079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:13.594563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1842c-9695-4219-830b-78f2e4844cb7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:13.627052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:13.628555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:13.628840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:13.629119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:13.635715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:13.637163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-b06d3d89-a475-489c-9cbf-cfaad880d895 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:13.651819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:13.653039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:13.653899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 591/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 11:00:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:13.658919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:13.670416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:13.755945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d051d-262b-4152-aa3d-61e565142ec8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:13.789370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:13.790025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:13.790310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:13.790593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:13.791336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:13.791646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:13.793014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b07eb-23fd-490c-8c44-684340610be5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:13.800921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 575/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Thu Jul 14 11:00:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:15.829862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5583-15fb-4042-856f-d383799c838e None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:15.830428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5583-15fb-4042-856f-d383799c838e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:15.830428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d5583-15fb-4042-856f-d383799c838e None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:15.832402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 592/1167] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:15.837745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091e07-81ee-453a-bc47-839507bcd43f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:15.838130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091e07-81ee-453a-bc47-839507bcd43f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:15.838308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091e07-81ee-453a-bc47-839507bcd43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:15.855369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-15091e07-81ee-453a-bc47-839507bcd43f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:15.856553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-15091e07-81ee-453a-bc47-839507bcd43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:15.921130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 576/1168] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:15.944704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d5583-15fb-4042-856f-d383799c838e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:16.010244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:16.044325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d5583-15fb-4042-856f-d383799c838e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:16.077821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:16.078472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:16.078755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:16.079027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:16.079797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:16.081887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599653a-d61e-4f9d-aabd-23443b1953d9 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:16.139937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 593/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Thu Jul 14 11:00:15 2022] POST /identity/v3/domains => generated 267 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:18.193905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580606-3e1b-4608-b8f6-14f16188dce8 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:18.194370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580606-3e1b-4608-b8f6-14f16188dce8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:18.194459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580606-3e1b-4608-b8f6-14f16188dce8 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:18.195866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 577/1170] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:18.201022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:18.201460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:18.201591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:18.218508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:18.219718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:18.283810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 594/1171] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:18.307037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580606-3e1b-4608-b8f6-14f16188dce8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:18.310783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:18.406601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580606-3e1b-4608-b8f6-14f16188dce8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:18.439388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:18.440848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:18.441126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:18.441401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] PATH_INFO: `/v3/domains/23185ef6ac404d6cb88dcbafbac6c2a6` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:18.451579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=23185ef6ac404d6cb88dcbafbac6c2a6)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:18.453022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9217117-69e3-4089-baec-2849f0ddaee6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:18.454644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 578/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Thu Jul 14 11:00:18 2022] GET /identity/v3/domains/23185ef6ac404d6cb88dcbafbac6c2a6 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:18.461260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:18.514203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:18.557544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e16f1b-750e-40d2-a574-e015b6d2e32f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:18.590520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:18.591206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:18.591498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:18.591778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:18.592485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:18.593922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e66d8-65d9-4850-98fd-de590a593c30 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:18.677617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 595/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 11:00:18 2022] POST /identity/v3/projects => generated 361 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:20.717783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c31ef-fea1-4d8c-911a-0202119aeedd None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:20.718365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c31ef-fea1-4d8c-911a-0202119aeedd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:20.718365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c31ef-fea1-4d8c-911a-0202119aeedd None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:20.719743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 579/1174] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:20.725500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:20.725902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:20.726067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:20.743170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:20.744337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:20.811848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 596/1175] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:20.837476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c31ef-fea1-4d8c-911a-0202119aeedd None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:20.903112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:20.942680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c31ef-fea1-4d8c-911a-0202119aeedd None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:20.975852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:20.976635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:20.976917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:20.977192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] PATH_INFO: `/v3/domains/23185ef6ac404d6cb88dcbafbac6c2a6` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:20.978339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=23185ef6ac404d6cb88dcbafbac6c2a6)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:20.979838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c3569-aa07-4d8b-961e-dcab454b0059 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:20.981426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 580/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Thu Jul 14 11:00:20 2022] GET /identity/v3/domains/23185ef6ac404d6cb88dcbafbac6c2a6 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:20.987487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:21.026688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:21.087334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cacf2-b80d-4240-80fb-025df737a509 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:21.119941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:21.120599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:21.120881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:21.121155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:21.121842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:21.123292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da33bf-8dd4-4a9b-a89e-4adea5c04615 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:21.225033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 597/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 11:00:20 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:23.302796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8689-5e94-452b-8690-28bd403cbd9b None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.303518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8689-5e94-452b-8690-28bd403cbd9b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.303518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce8689-5e94-452b-8690-28bd403cbd9b None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.304833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 581/1178] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:23.309999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.310417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.310562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.328255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:23.329421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:23.394948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 598/1179] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:23.417855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce8689-5e94-452b-8690-28bd403cbd9b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:23.516601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce8689-5e94-452b-8690-28bd403cbd9b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:23.545065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:23.550465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:23.551278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.551567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.551848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.561539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:23.569742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:23.570893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:00:23.571849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 582/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:23.576918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:23.600345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:23.673015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1ab7-0c1c-4810-8c7c-a2fe69bc20f6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:23.706720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:23.707403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.707687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.707959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.708693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:23.709000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:23.710356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:23.718163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 599/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:23.724076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:23.820926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0829865-f342-4e51-bcb7-e4abcebef0e5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:23.852943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:23.853798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.854086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.854365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] PATH_INFO: `/v3/users/1712d4ce884a4a5ea1faa717b12abfd2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.867667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] RBAC: Authorizing `identity:get_user(user_id=1712d4ce884a4a5ea1faa717b12abfd2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:23.869109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:23.870737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 583/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:23 2022] GET /identity/v3/users/1712d4ce884a4a5ea1faa717b12abfd2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:23.877222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:23.959373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:23.974080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1a9fb-5e8b-4527-8404-01536acc8f65 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:24.007048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:24.007706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:24.007987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:24.008260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:24.072343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:24.073714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:24.152194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:24.167398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 600/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:00:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:00:24.173039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:24.222874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:24.271319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a972db6-3077-4c01-b6b6-65aaad4d4dcd None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:24.304074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:24.304928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:24.305213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:24.305487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] PATH_INFO: `/v3/projects/b637435aa37e4453b5974cefb8fc54f6` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:24.306664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] RBAC: Authorizing `identity:get_project(project_id=b637435aa37e4453b5974cefb8fc54f6)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:24.308114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13990f7a-38d6-4b2d-93e3-f1d5101de87f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:24.309787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 584/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:24 2022] GET /identity/v3/projects/b637435aa37e4453b5974cefb8fc54f6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:24.316025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:24.412803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d24be-f16b-48eb-8e17-04665cee6859 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:24.445411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:24.446072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:24.446336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:24.446609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:24.447461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=b637435aa37e4453b5974cefb8fc54f6, user.id=1712d4ce884a4a5ea1faa717b12abfd2) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:24.447771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:24.449183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad63519-754e-4132-9e12-a15f73934434 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:24.454901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 601/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:00:24 2022] GET /identity/v3/role_assignments?user.id=1712d4ce884a4a5ea1faa717b12abfd2&scope.project.id=b637435aa37e4453b5974cefb8fc54f6&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:26.453025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9145d-2ae0-4f8e-b30d-0851af12f0c2 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.453534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9145d-2ae0-4f8e-b30d-0851af12f0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.453534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9145d-2ae0-4f8e-b30d-0851af12f0c2 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.455636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 585/1186] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:26.461049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.461478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.461602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.478807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:26.480019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.524800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:26.544316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 602/1187] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:26.568368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9145d-2ae0-4f8e-b30d-0851af12f0c2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.666188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9145d-2ae0-4f8e-b30d-0851af12f0c2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:26.683110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:26.699488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:26.700279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.700565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.700841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.711020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:26.720723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:26.721837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:00:26.722851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 586/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:26.728021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.824475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6fe6f-2105-4da6-8467-ec5cc7843610 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:26.856624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:26.857275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.857556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.857825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.858550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:26.858851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:26.860243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:26.868758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 603/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:26.874605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.973603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9c774-b09a-42b7-9bf5-b1217b0beccc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:27.005917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:27.006780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.007066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.007369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] PATH_INFO: `/v3/users/1712d4ce884a4a5ea1faa717b12abfd2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.008549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] RBAC: Authorizing `identity:get_user(user_id=1712d4ce884a4a5ea1faa717b12abfd2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:27.009978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:27.011627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 587/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:26 2022] GET /identity/v3/users/1712d4ce884a4a5ea1faa717b12abfd2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:27.017781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:27.060091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:27.116032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f251e-c143-475b-840c-9e06a8f1a360 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:27.148273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:27.149122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.149404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.149684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] PATH_INFO: `/v3/projects/b637435aa37e4453b5974cefb8fc54f6` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.150838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] RBAC: Authorizing `identity:get_project(project_id=b637435aa37e4453b5974cefb8fc54f6)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:27.152281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e1a1e-f700-4448-ad09-ca3616a01871 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:27.153940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 604/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:27 2022] GET /identity/v3/projects/b637435aa37e4453b5974cefb8fc54f6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:27.160384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:27.206556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:27.258693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e3b8-cc58-4553-bdc1-28a7160985aa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:27.291612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:27.292498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.292785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.293063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] PATH_INFO: `/v3/projects/b637435aa37e4453b5974cefb8fc54f6/users/1712d4ce884a4a5ea1faa717b12abfd2/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.294994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] RBAC: Authorizing `identity:create_grant(project_id=b637435aa37e4453b5974cefb8fc54f6, user_id=1712d4ce884a4a5ea1faa717b12abfd2, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:27.296460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adb9c5-a20c-4607-bf80-027f11eb4606 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:27.378641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 588/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 11:00:27 2022] PUT /identity/v3/projects/b637435aa37e4453b5974cefb8fc54f6/users/1712d4ce884a4a5ea1faa717b12abfd2/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:29.397592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a703f6-49d2-49e2-a582-10662143aacd None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.398141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a703f6-49d2-49e2-a582-10662143aacd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.398141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a703f6-49d2-49e2-a582-10662143aacd None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.399560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 605/1193] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:29.404670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.405048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.405233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.422281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:29.424052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:29.489821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 589/1194] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:00:29.513509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a703f6-49d2-49e2-a582-10662143aacd None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:29.612266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a703f6-49d2-49e2-a582-10662143aacd None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:29.622285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:29.645024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:29.645838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.646122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.646396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.656000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:00:29.665057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:00:29.666165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:00:29.667450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 606/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:00:29.672073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:29.705878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:29.769628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fba422-cc91-4a14-b1ef-067fa35e0aa3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:29.806535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:29.807236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.807529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.807810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.808558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:29.808871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:29.810240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:29.818463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 590/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:29.824162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:29.921034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f485f45-e559-45b5-a3bb-63f29cd9dedf None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:29.953276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:29.954731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.955020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.955304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] PATH_INFO: `/v3/users/1712d4ce884a4a5ea1faa717b12abfd2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.956483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] RBAC: Authorizing `identity:get_user(user_id=1712d4ce884a4a5ea1faa717b12abfd2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:29.957903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:29.959547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 607/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Thu Jul 14 11:00:29 2022] GET /identity/v3/users/1712d4ce884a4a5ea1faa717b12abfd2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:29.965382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:30.057773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:30.061494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0b1d-fdc0-4812-8e3a-9712588b41f7 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:30.096899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:30.097574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:30.097861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:30.098138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:30.168013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:30.169463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:30.261890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:30.267005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 591/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:00:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:00:30.272846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:30.312120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:30.370821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6e1c7-6211-442f-9e00-62bded997367 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:30.404286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:30.405175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:30.405457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:30.405729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] PATH_INFO: `/v3/projects/b637435aa37e4453b5974cefb8fc54f6` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:30.407088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] RBAC: Authorizing `identity:get_project(project_id=b637435aa37e4453b5974cefb8fc54f6)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:30.408549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987df249-dbbd-44c2-86b4-fe99f55c051a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:30.410231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 608/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Thu Jul 14 11:00:30 2022] GET /identity/v3/projects/b637435aa37e4453b5974cefb8fc54f6 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:30.416571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:30.514866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5de06-8306-437e-8951-eef0ee0a6f9c None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:30.548168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:30.548818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:30.549105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:30.549387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:30.550207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=b637435aa37e4453b5974cefb8fc54f6, user.id=1712d4ce884a4a5ea1faa717b12abfd2) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:30.550520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:30.551967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3adb9-2d57-45f7-8e28-eaa361058915 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:30.557615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 592/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:00:30 2022] GET /identity/v3/role_assignments?user.id=1712d4ce884a4a5ea1faa717b12abfd2&scope.project.id=b637435aa37e4453b5974cefb8fc54f6&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:40.400269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=47 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:40.555000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=69 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:42.494522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022a7f-01f2-480d-8fe6-9839ed3bd12a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:42.495087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022a7f-01f2-480d-8fe6-9839ed3bd12a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:42.495087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24022a7f-01f2-480d-8fe6-9839ed3bd12a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:42.496544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 609/1201] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:42.501866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2312160-6636-402e-999e-4fe285a8761e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:42.502291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2312160-6636-402e-999e-4fe285a8761e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:42.502429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2312160-6636-402e-999e-4fe285a8761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:42.519885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a2312160-6636-402e-999e-4fe285a8761e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:42.521577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a2312160-6636-402e-999e-4fe285a8761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:42.588790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 593/1202] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:01:42.613605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24022a7f-01f2-480d-8fe6-9839ed3bd12a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:42.713228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24022a7f-01f2-480d-8fe6-9839ed3bd12a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:42.746832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:42.747772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:42.748061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:42.748337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:42.758119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:42.759846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:42.776932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:42.778061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:42.780117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:42.780117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:42.780117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:42.780117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:42.780117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 610/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:42.784722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2312160-6636-402e-999e-4fe285a8761e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:42.882644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2312160-6636-402e-999e-4fe285a8761e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:42.911315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:42.916999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:42.917654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:42.917939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:42.918216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:42.918936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:42.919983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:42.921596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045dc5f-4806-4a35-bdc4-5dbc5c922ab6 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:42.932904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 594/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:42.938984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:42.991267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:43.036834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4ca10-3899-4fb7-888b-7938bf08dd68 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:43.068256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:43.068916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:43.069206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:43.069483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:43.070184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:43.071650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0347fb-9af5-47a7-b9d8-40be443eac5f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:43.160138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 611/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Thu Jul 14 11:01:42 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:45.273886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109817c-e8e7-4140-8448-eea2a700064a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:45.274471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109817c-e8e7-4140-8448-eea2a700064a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:45.274869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109817c-e8e7-4140-8448-eea2a700064a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:45.276440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 595/1206] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:45.282742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bf33e-5206-49f5-861c-977743310933 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:45.283233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bf33e-5206-49f5-861c-977743310933 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:45.283333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bf33e-5206-49f5-861c-977743310933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:45.305081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-029bf33e-5206-49f5-861c-977743310933 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:45.306345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-029bf33e-5206-49f5-861c-977743310933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:45.381739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 612/1207] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:01:45.410059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109817c-e8e7-4140-8448-eea2a700064a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:45.523830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109817c-e8e7-4140-8448-eea2a700064a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:45.561015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748274b-c266-401c-af41-486866fedfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:45.561819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748274b-c266-401c-af41-486866fedfbb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:45.562104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748274b-c266-401c-af41-486866fedfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:45.562379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748274b-c266-401c-af41-486866fedfbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:45.572561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748274b-c266-401c-af41-486866fedfbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-f748274b-c266-401c-af41-486866fedfbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:01:45.582649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:45.583812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:45.584856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 596/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 11:01:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:45.592605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029bf33e-5206-49f5-861c-977743310933 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:45.633172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:45.691857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029bf33e-5206-49f5-861c-977743310933 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:45.725372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:45.726047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:45.726336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:45.726613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:45.727380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:45.727691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:45.729054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:45.737906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 613/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 11:01:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:45.743839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748274b-c266-401c-af41-486866fedfbb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:45.796986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:45.846914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748274b-c266-401c-af41-486866fedfbb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:45.882143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:45.882959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:45.883273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:45.883550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:45.894410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:45.895912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:45.913612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:45.914706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:45.915868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:45.915868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:45.915868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:45.915868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:45.915868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 597/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:45.921370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.022187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:46.025130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943fd31-e073-4420-b50e-20df67183e2f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:46.058132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:46.058729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:46.059016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:46.059352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:46.060069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:46.060378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:46.061778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:46.073058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 614/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:46.078927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.179068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1c39-fa8b-4a7f-a865-3a4fb1875efb None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:46.180729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:46.215423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:46.217395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:46.217676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:46.218004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:46.225265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:46.228415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:01:46.241686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:46.242856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:46.243842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:46.244575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 598/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:01:46.249673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.351949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e57bb-f280-4a91-9299-c9e43e2e676b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:46.386356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:46.387053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:46.387370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:46.389257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:46.389257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:46.389257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:46.390180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:46.407601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 615/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:01:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:46.413369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.514060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e85fa-1430-4059-8d08-acc6177693dc None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:46.548337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:46.548982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:46.549272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:46.549553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:46.579958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:46.617754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:46.619299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:46.715603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 599/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:01:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:01:46.721623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.740980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:46.820666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5b698-b2c3-486f-99d5-8730dcdf4b1a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:46.855414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:46.856774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:46.857171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:46.857561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:46.867544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:46.869008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:46.886943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:46.892182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:46.893217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:46.893217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:46.893217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:46.893217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:46.893217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 616/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:46.894043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:46.955145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:46.993920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45c1b-79a6-43d2-9424-00e3113a2962 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:47.030051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:47.030714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:47.031013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:47.031365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:47.032085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:47.032393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:47.033763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:47.046015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 600/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:47.051719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:47.119646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:47.151047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d8a48-62ee-4571-890c-4c8b1f9cb181 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:47.185556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:47.192887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:47.192887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:47.192887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:47.193610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:47.195050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:01:47.209005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:47.209954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:47.210847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 617/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:47.216603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:47.314170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6d0fa-5bf6-485c-9cbd-f5e068a288fa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:47.336249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:47.348069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:47.348726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:47.349011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:47.349288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:47.350075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:47.350388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:47.351866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2896197-c13d-4672-974c-313847c2bf99 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:47.364130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 601/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:47.370214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:47.471019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd002df-703b-46e4-98f9-9c631fbd668e None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:47.499502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:47.504952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:47.505602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:47.505881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:47.506152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:47.506971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=aaf80b4722094475934134be619652e1) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:47.507312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:47.510995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa666b-da29-4f54-9a64-885a331a6a0f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:47.515116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 618/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:01:47 2022] GET /identity/v3/role_assignments?user.id=aaf80b4722094475934134be619652e1&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:49.544561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d2d-2621-4420-924c-c899fd7eed98 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:49.545114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d2d-2621-4420-924c-c899fd7eed98 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:49.545114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d2d-2621-4420-924c-c899fd7eed98 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:49.546494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 602/1220] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:49.551981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:49.552432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:49.552581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:49.569563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:49.570767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:49.636053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 619/1221] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:01:49.659679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18610d2d-2621-4420-924c-c899fd7eed98 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:49.760085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18610d2d-2621-4420-924c-c899fd7eed98 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:49.793176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:49.794763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:49.794992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:49.795297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:49.805268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:01:49.813882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:49.815108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:49.816199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 603/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 11:01:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:49.821412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:49.917276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db7eba-aaf5-40d9-923c-5ab835eb2dcc None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:49.941568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:49.952035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:49.952695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:49.952979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:49.953253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:49.953999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:49.954307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:49.955720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:49.963574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 620/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 11:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:49.969789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.014112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:50.070008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce5784-e983-418c-b4e4-648f4de5fc69 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.103029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.103842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.104127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.104405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.115404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.116852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:50.133721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:50.134815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:50.136047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:50.136047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:50.136047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:50.136047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:50.136047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 604/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:50.140970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.240627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510bdb-fae5-4ec5-85d3-edf44c39eee2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.274159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.274820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.275106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.275418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.276133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:50.276443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.277876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.289618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 621/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:50.296091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.390616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:50.397542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5264e1-b48d-457c-80e1-06a133a93082 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.434792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.435605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.435892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.436169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.443668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.445275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:01:50.460597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:50.461577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:50.462611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 605/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:50.468346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.515296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:50.567312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993d89d-ab38-4c1f-8876-73555d9762bd None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.600655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.601326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.601614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.601892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.602720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:50.603044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.604488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.628031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 622/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:50.634249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.738331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e86f93-3309-4d6a-ab8d-cebe9eca5f9b None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.772052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.772879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.773162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.773434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.784216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.785705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:50.803839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:50.807674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:50.808842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:50.808842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:50.808842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:50.808842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:50.808842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 606/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:50.811788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.917589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675eb22-ab33-4d15-a8b9-0d7451352882 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.920671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:50.952063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.952721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.953002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.953277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.953988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:50.954292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.955686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.967506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 623/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:50.973240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.982828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:51.070623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689076e1-c966-4dc2-9555-1caeb52d6998 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:51.104759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:51.105551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:51.106664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:51.106859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:51.113460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:51.114902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:01:51.129020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:51.129950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:51.131028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 607/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:51.136427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:51.238006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560ceaa-f334-4cc5-9b7d-93aa3997bc76 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:51.275285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:51.275971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:51.276288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:51.276596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:51.277385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:51.278418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:51.279891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208db4e-76cd-47b6-a1f3-82734ffa2f04 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:51.291608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 624/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:51.297738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:51.363816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:51.399307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb751770-b10f-4f0e-ab83-451f5f66da0d None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:51.433029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:51.434458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:51.434746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:51.435024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/aaf80b4722094475934134be619652e1/roles/e2afff0579ef41c8a45043e5e7d93a07` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:51.448650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=aaf80b4722094475934134be619652e1, role_id=e2afff0579ef41c8a45043e5e7d93a07)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:51.451133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e083bc-38ca-43c3-9cad-9079c3054f09 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:51.576700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 608/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 11:01:51 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/aaf80b4722094475934134be619652e1/roles/e2afff0579ef41c8a45043e5e7d93a07 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:53.622869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947b50-342c-4965-8bee-d9f2216ccae1 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.623452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947b50-342c-4965-8bee-d9f2216ccae1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.623452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947b50-342c-4965-8bee-d9f2216ccae1 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.624878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 625/1233] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:53.630183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.630599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.630740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.648311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:53.649505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:53.717115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 609/1234] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:01:53.741485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82947b50-342c-4965-8bee-d9f2216ccae1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:53.766874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:53.842684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82947b50-342c-4965-8bee-d9f2216ccae1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:53.876595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:53.877329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.877613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.877886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.888425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:01:53.896090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:53.897239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:53.898358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 626/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Thu Jul 14 11:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:53.903927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:53.949962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.001143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff57f7-1ecc-441e-afc3-cfd9b66d1d65 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.034653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.035724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.035724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.035905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.036631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] RBAC: Adding query filter params (name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:54.036935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.038291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.046642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 610/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Thu Jul 14 11:01:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:54.052315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.136303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.151404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b650-cddf-48ea-b6f3-7eff7a16c41b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.186231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.187029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.187340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.187616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.197587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.199787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:54.218257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:54.219411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:54.220347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:54.220347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:54.220347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:54.220347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:54.220347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 627/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:54.225402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.325051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1abe6-3fdf-4176-8af9-33b14c35e908 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.333721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.358007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.358669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.358952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.359257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.359979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:54.360285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.361653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.373286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 611/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:54.379065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.478413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7126f-fe2b-49c7-8e59-f50175087470 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.512065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.513105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.513387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.513673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.516438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.520881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.522335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:01:54.538252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:54.543434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:54.544596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 628/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:54.545489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.648579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37323d-697c-494e-9476-bbd476e58c4a None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.685400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.686112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.686362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.686639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.687522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:54.687835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.689264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.706292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 612/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:01:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:54.712179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.810929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a797b-49b6-49cd-8c0b-6e848198df4f None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.844823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.845474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.845762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.846040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.906733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.913086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.914548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.014674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 629/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:01:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:01:55.020771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.048655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.117558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508cfaf-f72d-4060-8497-f85476bbf813 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.155433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.156228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.156511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.156790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.166784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.168882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:55.187621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:55.188718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:55.189745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:55.189745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:55.189745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:55.189745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:55.189745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 613/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:55.194661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.286061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.291831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf4565-c4fe-4ef4-8080-6e30ad40d18a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.324928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.325585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.325878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.326153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.326869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:55.327203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.328574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.340680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 630/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:01:55.346532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.426713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.447013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c5e9-0397-4caa-ac48-969864ad0233 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.481102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.481890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.482179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.482456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.489019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.490460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:01:55.503964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:55.504944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:55.505832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 614/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 11:01:55.511409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.611824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012aeb-34ee-46f4-9a69-c91aa82d8949 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.644530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.645183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.645475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.645747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.646520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:55.646830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.648427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62682f9f-ffee-4cf8-8c4d-4bbf2cd8ff85 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.660896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 631/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:01:55.666922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.766713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241a73-3796-42c8-bdd4-a2f24ec1f1c3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.800063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.800717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.801003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.801282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.802136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] RBAC: Adding query filter params (role.id=e2afff0579ef41c8a45043e5e7d93a07, scope.project.id=86229daf0943484393a30086a3139985, user.id=aaf80b4722094475934134be619652e1) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:55.802449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.803909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78dce6-d7ab-41d4-8640-83282291b146 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.809727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 615/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/role_assignments?user.id=aaf80b4722094475934134be619652e1&scope.project.id=86229daf0943484393a30086a3139985&role.id=e2afff0579ef41c8a45043e5e7d93a07 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:57.902535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea4fc-5ba3-488a-86ce-9dc8a71edd35 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:57.903188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea4fc-5ba3-488a-86ce-9dc8a71edd35 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:57.903295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea4fc-5ba3-488a-86ce-9dc8a71edd35 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:57.904665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 632/1247] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:57.909843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:57.910267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:57.910403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:57.915419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:57.927826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:57.929140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:57.997897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 616/1248] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:01:58.022188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea4fc-5ba3-488a-86ce-9dc8a71edd35 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.040374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:58.125074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea4fc-5ba3-488a-86ce-9dc8a71edd35 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.158827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.159579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.159871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.160154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.171105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:01:58.178765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:58.179880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:58.180930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 633/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:58.189369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.291851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd7eef-c8cc-4c7f-8fff-6a43f402c4a9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.320625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:58.325803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.326466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.326748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.327022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.327794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:58.328102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:58.329472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:58.337295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 617/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:58.343960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.421926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:58.444408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f162c0-0dfd-4bcf-8ca3-3e954fe95f93 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.477186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.477973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.478256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.478529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.489476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:58.490919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:58.508234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:58.513239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:58.514325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:58.514325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:58.514325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:58.514325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:58.514325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 634/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:58.516094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.616487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3d9f1-db65-41bc-8808-146ab67cc8f5 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.649505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.650161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.650446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.650726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.651471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:58.651781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:58.653744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:58.665084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 618/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:58.670919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.772140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dfede-6ab7-4993-b427-ed51aa731c3b None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.798998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:58.806538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.807364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.807644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.807921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.815155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:58.816604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:01:58.831273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:58.832546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:01:58.833690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 635/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:58.838242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:58.896133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:58.943301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f558d8-2066-46b7-93ca-ebf2d38d2795 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:58.977553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:58.978224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:58.978513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:58.978790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:58.979697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:58.980012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:58.982050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.000154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 619/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:01:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:59.005980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.106948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caaf95-9ee3-4d19-a073-d317a979aa19 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:59.139495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:59.140145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:59.140432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:59.140710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:59.196810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:59.209492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:59.210977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.307630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 636/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:01:59.314010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.413080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4df5b-eee3-47b3-a45f-72cfe02f8185 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:59.426870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:59.447773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:59.448597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:59.448879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:59.449150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:59.459270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:59.460745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:01:59.477901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:59.478973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:59.483295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:01:59.483295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:01:59.483295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:01:59.483295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:59.483295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 620/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:59.485569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.573606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:59.585362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee56f8b-c7ce-42f4-a0d0-a05553a1a1bf None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:59.618818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:59.619537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:59.619800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:59.620102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:59.620820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:59.621249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:59.623176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.635097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 637/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:59.641141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.741030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6ae7-6d8e-4819-a1cf-cc924ed9aeb1 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:59.774866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:59.775677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:59.775963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:59.776236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:59.783749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:59.785234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:01:59.799927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:01:59.800836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:01:59.801830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 621/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:01:59.806992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.906620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4dd3b-8cac-475f-acbf-56c1548f4af4 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:59.939284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:59.940264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:59.940264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:59.940518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:59.941292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:59.941601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:59.943020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbf8ef-097e-4a2a-aad4-1fb3ac121abc None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:59.955315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 638/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:59.961372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:59.987615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:00.064921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd142b83-db2d-4e0b-a02d-02cfeaae76f2 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:00.098977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:00.099661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:00.099956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:00.100231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:00.101066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=aaf80b4722094475934134be619652e1) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:00.101383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:00.102823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103df2a-a4f2-464d-8079-91fa5aeb1e4f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:00.108284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 622/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:01:59 2022] GET /identity/v3/role_assignments?user.id=aaf80b4722094475934134be619652e1&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:02.131826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346052-b748-4bf9-990f-9cde002e6183 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.132728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346052-b748-4bf9-990f-9cde002e6183 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.132728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346052-b748-4bf9-990f-9cde002e6183 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.133999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 639/1261] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:02.139538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.139947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.140099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.157077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:02.158264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:02.228607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 623/1262] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:02.252315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6346052-b748-4bf9-990f-9cde002e6183 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:02.263497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:02.352956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6346052-b748-4bf9-990f-9cde002e6183 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:02.387292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:02.388012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.388293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.388563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.398486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:02:02.407271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:02.414887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:02.415848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 640/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:02.416562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:02.434589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:02.513614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662bdbd-0567-43b7-b78f-e8c46df8c9d3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:02.548775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:02.549442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.549748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.550023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.550770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:02.551077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:02.552469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:02.561722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 624/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:02:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:02.567638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:02.634476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:02.667441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4af9c-7b66-436d-904c-b5883331fc86 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:02.701953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:02.702737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.703024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.703330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.713784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:02.715492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:02:02.733162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:02.739354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:02.740333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:02:02.740333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:02:02.740333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:02.740333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:02.740333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 641/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:02.740618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:02.822354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:02.841770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863182-96d4-43b1-9a61-fe6fff8f2d44 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:02.875352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:02.876013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:02.876296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:02.876572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:02.877322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:02.877628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:02.878995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:02.890959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 625/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:02.896745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:02.998812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa30bf-4069-4a1e-befa-5431d12fe5ee None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.019738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:03.033728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.034529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.034818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.035151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.043531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.044985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:02:03.059750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:03.060867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:03.061984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 642/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:03.066767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.165455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbff7b-7cfd-4858-8b6c-5bdd1dfe8568 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.205326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.205992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.206275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.206548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.207411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:03.207721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.209202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.218464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:03.226224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 626/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:02:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:03.232728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.331727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563279ad-3b39-4f5d-b3f6-dcf579db8228 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.365505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.366280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.366560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.366834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.378178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.379638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:02:03.396631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:03.397520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:03.398563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:02:03.398563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:02:03.398563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:03.398563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:03.398563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 643/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:03.403844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.504073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef4c08-6e4e-4204-bb58-c4a3900d92db None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.538036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.538765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.539055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.539361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.540103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:03.540419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.541812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.553591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 627/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:03.559948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.592521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:03.663366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0942-7e8f-42e2-8934-242ce3e99212 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.697733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.698515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.699970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.700281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.706924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.708396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:02:03.722599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:03.723677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:03.725097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 644/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:03.729905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.788859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:03.832096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d7af9-d02c-4ead-90f2-d8baf2df2115 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:03.865254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:03.865839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:03.866123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:03.866405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:03.867207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:03.867518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:03.868948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a02a0-95b6-4e3c-b518-4c9947816a0c None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:03.881423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 628/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:03.888551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:03.983876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:03.992500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5cdc-975b-4ba8-864b-24a89774b39a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:04.026789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:04.027776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:04.028059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:04.028330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] PATH_INFO: `/v3/projects/86229daf0943484393a30086a3139985/users/aaf80b4722094475934134be619652e1/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:04.030304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] RBAC: Authorizing `identity:create_grant(project_id=86229daf0943484393a30086a3139985, user_id=aaf80b4722094475934134be619652e1, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:04.031752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a57a0-5e63-48fc-a5a9-fd7f28b70815 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:04.111944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 645/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Thu Jul 14 11:02:03 2022] PUT /identity/v3/projects/86229daf0943484393a30086a3139985/users/aaf80b4722094475934134be619652e1/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:06.126827 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd051758-a559-46f5-afee-0fc2d1bef492 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.127433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd051758-a559-46f5-afee-0fc2d1bef492 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.127433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd051758-a559-46f5-afee-0fc2d1bef492 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.128803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 629/1274] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:06.134294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.134710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.134858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.151907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:06.153104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:06.220414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 646/1275] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:06.246138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd051758-a559-46f5-afee-0fc2d1bef492 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.346666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd051758-a559-46f5-afee-0fc2d1bef492 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.364488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:06.381676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.382666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.382951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.383264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.393765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 11:02:06.401406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:06.402882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:06.409516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 630/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:06.410375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.490200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:06.508871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36476816-a440-4051-b927-9cca20f5ad72 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.542109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.543406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.543696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.544005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.544739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:06.545044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.546406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.554425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 647/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:06.560420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.660249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a08d36-3d00-4345-a380-04f8ddd9f137 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.697066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.697854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.698223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.698498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.709414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.715227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:02:06.728760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:06.729821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:06.730715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:02:06.730715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:02:06.730715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:06.730715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:06.730715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 631/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:06.736554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.836143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a0282-947a-4694-9215-01581b416795 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.869473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.870133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.871094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.871410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.872132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:06.872444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.873844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.885586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 648/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:06.891999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.923290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:06.990445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfcb5e-8042-4e85-a210-c905dc082ade None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.024287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.025094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.025381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.025657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.033629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.035073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 11:02:07.050095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:07.051308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:07.052371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 632/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:07.057095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.061400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:07.154178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365797-d275-422b-a192-c11cc2d83087 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.188652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.189328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.189612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.189889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.190794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:07.191106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.192601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.209872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 649/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:07.215746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.307716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:07.316909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa810baa-06a6-4e29-9403-02d6de2e5ec9 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.350974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.351643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.351928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.352204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.421022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.422573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.524768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:07.536433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 633/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 11:02:07.542502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.611359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:07.653474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92662420-3619-4df3-b51e-4f0f13ba9317 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.691089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.691934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.692225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.692507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.704156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.705665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 11:02:07.724651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:07.725812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:07.726799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 11:02:07.726799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 11:02:07.726799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 11:02:07.726799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:07.726799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 650/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:07.732620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.841056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceb78a-f32d-44c7-b3db-6e352768b064 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.877077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.884979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.892468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 634/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:07.898227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.998094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f66b8-dbe3-4893-a6b0-330eed5745e2 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.011377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:08.031893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.032708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.032991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.033264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.039721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.041164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 11:02:08.056065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:08.057161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: ERROR keystone.server.flask.application Jul 14 11:02:08.058325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 651/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:08.063516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.113540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:08.162466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13254da3-7362-40e1-a5c5-7ebdddf14b15 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.196477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.197145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.197435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.197715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.198639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:08.198955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.200427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e4d2f-e5a5-4389-8d61-0e97a94d105f None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.212499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 635/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Thu Jul 14 11:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:08.219490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.320691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb200d-ceb1-47fa-8fac-229eecea8c8a None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.355747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.356430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.356713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.356987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.357853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] RBAC: Adding query filter params (role.id=beb4a709a14b415db991ad1980b406a9, scope.project.id=86229daf0943484393a30086a3139985, user.id=aaf80b4722094475934134be619652e1) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:08.358208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.359645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddab6c-8493-438f-9d2e-beedc42e107d None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.365431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 652/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Thu Jul 14 11:02:08 2022] GET /identity/v3/role_assignments?user.id=aaf80b4722094475934134be619652e1&scope.project.id=86229daf0943484393a30086a3139985&role.id=beb4a709a14b415db991ad1980b406a9 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:10.418256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80c51-338d-47bd-816c-e59609670049 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.418809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80c51-338d-47bd-816c-e59609670049 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.418809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80c51-338d-47bd-816c-e59609670049 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.420215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 636/1288] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:10.425427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.425885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.425998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.443561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:10.444688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:10.474387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:10.512476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 653/1289] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:10.535069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa80c51-338d-47bd-816c-e59609670049 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.637103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa80c51-338d-47bd-816c-e59609670049 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.647705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:10.670892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.671727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.672018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.672299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.673047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.674446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:10.686597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:10.691109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:10.692171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 637/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:10.694027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.790951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af68ac5-e7e3-4841-b565-f2ddada3d0c8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.824506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.825189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.825473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.825749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.826441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:10.826747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.828135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ea812-5fe3-41c0-a9f0-dfb842035319 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.832877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 654/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:10.838986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.939353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49f120-03b5-48e2-b76c-da72ac65d1fa None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.972378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.972976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.973257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.973532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] PATH_INFO: `/v3/services` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.974221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] RBAC: Adding query filter params (type=placement) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:10.974526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.975928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec236a2-cbb9-47b6-9bca-10aeed9aa8ec None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.981129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 638/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:13.009242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f5fea-9b33-4456-b37d-b48eacfe87b8 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:13.009796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f5fea-9b33-4456-b37d-b48eacfe87b8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:13.009796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f5fea-9b33-4456-b37d-b48eacfe87b8 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:13.011832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 655/1293] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:13.016940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afab7d-58a6-46cc-87e4-2b6a1d3ff804 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:13.021205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afab7d-58a6-46cc-87e4-2b6a1d3ff804 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:13.021205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afab7d-58a6-46cc-87e4-2b6a1d3ff804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:13.034019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c3afab7d-58a6-46cc-87e4-2b6a1d3ff804 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:13.035227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c3afab7d-58a6-46cc-87e4-2b6a1d3ff804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:13.066867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:13.102437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 639/1294] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:13.126537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f5fea-9b33-4456-b37d-b48eacfe87b8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:13.157520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:13.227638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f5fea-9b33-4456-b37d-b48eacfe87b8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:13.262930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:13.263596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:13.263873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:13.264145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:13.264861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:13.266212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfceed1-120e-48ad-945a-af8b5da57e4a None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:13.339771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 656/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Thu Jul 14 11:02:13 2022] POST /identity/v3/services => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:15.354987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec010f5-b96e-4bf6-b07b-37fad46cc1d7 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:15.355632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec010f5-b96e-4bf6-b07b-37fad46cc1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:15.355632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec010f5-b96e-4bf6-b07b-37fad46cc1d7 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:15.357041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 640/1296] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:15.362237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:15.362662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:15.362788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:15.380121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:15.381549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:15.453956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 657/1297] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:15.476633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec010f5-b96e-4bf6-b07b-37fad46cc1d7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:15.578789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec010f5-b96e-4bf6-b07b-37fad46cc1d7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:15.613017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:15.613813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:15.614098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:15.614375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:15.615025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:15.616424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:15.628065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:15.631318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:15.632351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 641/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 11:02:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:15.635598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:15.653865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:15.732873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4167df-5652-4dd4-8513-98780e82fe5b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:15.765160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:15.765818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:15.766101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:15.766378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:15.767079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:15.767425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:15.768794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:15.773950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 658/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Thu Jul 14 11:02:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:15.780440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:15.812471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:15.879312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54cea3-3b5d-4d5f-bd1a-e7751619c0f8 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:15.913010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:15.916600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:15.916600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:15.916600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:15.916600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] RBAC: Adding query filter params (interface=public, service_id=1fa381f718fa42469ef296f46de7daff, region_id=RegionOne) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:15.916600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:15.917397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f67b6c-3c10-4788-8505-c586eb79a8f0 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:15.922459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 642/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Thu Jul 14 11:02:15 2022] GET /identity/v3/endpoints?service_id=1fa381f718fa42469ef296f46de7daff&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:15.928180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:16.009651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:16.027658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0252e8-e1f1-4975-8dd3-c5321f429247 None admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:16.063276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:16.063969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:16.064253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:16.064525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:16.065204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:16.066604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8647ef-302e-4718-b02b-d333b4d188c0 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:16.073781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 659/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Thu Jul 14 11:02:15 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:02:18.075479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854843d9-ee2a-46dc-9668-2b42e27efc25 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:18.076030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854843d9-ee2a-46dc-9668-2b42e27efc25 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:18.076030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854843d9-ee2a-46dc-9668-2b42e27efc25 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:18.077453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 643/1302] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:18.082407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:18.082830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:18.082965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:18.100056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:18.101249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:18.167848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 660/1303] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:02:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 11:02:18.190633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854843d9-ee2a-46dc-9668-2b42e27efc25 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:18.285450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:18.292448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854843d9-ee2a-46dc-9668-2b42e27efc25 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:18.325250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:18.326109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:18.326327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:18.326598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:18.327276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:18.328653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 11:02:18.341212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 11:02:18.342327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application with Lock( Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self._enter() Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application return self.creator() Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application created_value = creator( Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: ERROR keystone.server.flask.application Jul 14 11:02:18.343250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 644/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Thu Jul 14 11:02:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 11:02:18.348587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:18.424142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:18.446108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04301-4716-4a2e-96d1-4feb9f76df05 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:18.480130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:18.480792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:18.481112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:18.481588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] PATH_INFO: `/v3/services` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:18.482438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:18.483171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:18.484556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed401c5-87d2-41ca-98ef-613011b46a47 None admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:18.489614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 661/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Thu Jul 14 11:02:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:18.495575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:18.593559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f35086-6796-4de2-b784-8aa83da248b3 None admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:18.626087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:18.626738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:18.627023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:18.627327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:18.628076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:18.629452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428d2bd-3814-46b5-922f-824aa15c9afa None admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:18.810598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 645/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Thu Jul 14 11:02:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:02:28.486552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=30 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:28.803682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:20.668692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16f74-43d4-44d2-8615-5f20717dfe5c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:20.669146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16f74-43d4-44d2-8615-5f20717dfe5c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:20.669248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16f74-43d4-44d2-8615-5f20717dfe5c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:20.679792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac33a-550d-48ae-9117-471491ab980b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:20.680973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac33a-550d-48ae-9117-471491ab980b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:20.681636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac33a-550d-48ae-9117-471491ab980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:20.683999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 662/1307] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:20.833551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6fdac33a-550d-48ae-9117-471491ab980b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:20.837444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6fdac33a-550d-48ae-9117-471491ab980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:20.975761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 646/1308] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:21.025815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:21.026103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:21.026572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:21.028074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 663/1309] 10.222.0.59 () {54 vars in 964 bytes} [Thu Jul 14 11:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:21.034658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa765791-f2bf-4ea2-8830-f61e011f3993 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:21.034957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa765791-f2bf-4ea2-8830-f61e011f3993 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:21.035272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa765791-f2bf-4ea2-8830-f61e011f3993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:21.120304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-aa765791-f2bf-4ea2-8830-f61e011f3993 None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:21.121508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-aa765791-f2bf-4ea2-8830-f61e011f3993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:21.147072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:21.207981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 647/1310] 10.222.0.59 () {58 vars in 1061 bytes} [Thu Jul 14 11:08:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:21.217227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:21.219068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:21.338433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa0f60-7f52-4bea-9ca3-75d9e74e1ff6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:21.416315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:21.416886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:21.417170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:21.417446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:21.419078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:21.474923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:21.531374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:21.532827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348c32f-deba-4218-92ec-27d264d15cc7 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:21.553046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 664/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:23.928338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84445827-bfec-4a28-a286-e7da35aaf308 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:23.929501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84445827-bfec-4a28-a286-e7da35aaf308 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:23.929501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84445827-bfec-4a28-a286-e7da35aaf308 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:23.931361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 648/1312] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:23.936355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85bb1a-050a-407e-8999-7cd318b3146f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:23.936767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85bb1a-050a-407e-8999-7cd318b3146f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:23.936964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85bb1a-050a-407e-8999-7cd318b3146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.027458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8c85bb1a-050a-407e-8999-7cd318b3146f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:24.028683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8c85bb1a-050a-407e-8999-7cd318b3146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.133546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 665/1313] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:24.193854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.194146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.194482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.196114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 649/1314] 10.222.0.59 () {54 vars in 964 bytes} [Thu Jul 14 11:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:24.202752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92330fb-1e90-48af-89bf-f418cc11e9a2 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.203047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92330fb-1e90-48af-89bf-f418cc11e9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.203357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92330fb-1e90-48af-89bf-f418cc11e9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.214713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:24.289274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b92330fb-1e90-48af-89bf-f418cc11e9a2 None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:24.290579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b92330fb-1e90-48af-89bf-f418cc11e9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.375257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 666/1315] 10.222.0.59 () {58 vars in 1061 bytes} [Thu Jul 14 11:08:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:24.384922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:24.386514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.500089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a97b82-d07d-4183-83e3-3a55d685b6ef None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:24.572141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:24.574735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:24.575424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.575710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.575991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.577632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.691856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:24.693328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453d70b-516b-4963-9ded-4ed9c84853d1 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:24.713393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 650/1316] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:27.513259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:27.513807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:27.513807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:27.515217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 667/1317] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:27.520315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39528e-9310-4e15-b335-2c00ee79a680 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:27.520716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39528e-9310-4e15-b335-2c00ee79a680 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:27.520883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39528e-9310-4e15-b335-2c00ee79a680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:27.585476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ce39528e-9310-4e15-b335-2c00ee79a680 None None] MFA Rules not processed for user `a5a53c91bfd74877b8a19925a7e8da30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:27.586771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ce39528e-9310-4e15-b335-2c00ee79a680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:27.637934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:27.682036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 651/1318] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:27.721208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:27.722707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:27.825276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2072c12-a925-4508-9502-e9d76a842edd None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:27.828024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:27.898092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:27.898940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:27.899261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:27.899551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:27.901167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:28.008409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:28.010117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2b711-26ce-4564-a508-c082172b8c26 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:28.029985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 668/1319] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:30.919591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:30.920184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:30.920184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:30.921530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 652/1320] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:30.926613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cef8b-8d97-4637-b8d7-f807d6be5fce None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:30.927014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cef8b-8d97-4637-b8d7-f807d6be5fce None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:30.927201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cef8b-8d97-4637-b8d7-f807d6be5fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:30.966539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:30.990284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9e3cef8b-8d97-4637-b8d7-f807d6be5fce None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:30.992109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9e3cef8b-8d97-4637-b8d7-f807d6be5fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:31.090491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 669/1321] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:31.129629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:31.131220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:31.236325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8bc18-5ebe-4070-838c-c17d22de8954 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:31.311032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:31.311712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:31.312000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:31.312281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:31.313948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:31.320930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:31.421828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:31.423308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25a637-cbae-4e92-9af3-0d66aa254264 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:31.442638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 653/1322] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:34.542490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:34.543053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:34.543185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:34.544566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 670/1323] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:34.549808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee342c-e605-4bf0-995c-fedaa0766a71 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:34.550208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee342c-e605-4bf0-995c-fedaa0766a71 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:34.550491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee342c-e605-4bf0-995c-fedaa0766a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:34.615598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8bee342c-e605-4bf0-995c-fedaa0766a71 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:34.616996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8bee342c-e605-4bf0-995c-fedaa0766a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:34.681650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:34.729897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 654/1324] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:34.766641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:34.768267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:34.881004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ee565-e45c-4d0d-999f-5646a5b476b3 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:34.894325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:34.955570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:34.956222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:34.956501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:34.956772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:34.958394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:35.067698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.069387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88014b-1973-4d50-b515-e36ebd08247b service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.087549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 671/1325] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:37.198152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:37.198686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:37.198686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:37.200105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 655/1326] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:37.205295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e943-ac03-46ca-88ad-d865bca33e91 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:37.205685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e943-ac03-46ca-88ad-d865bca33e91 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:37.205845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e943-ac03-46ca-88ad-d865bca33e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:37.253629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:37.272109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0e60e943-ac03-46ca-88ad-d865bca33e91 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:37.273332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0e60e943-ac03-46ca-88ad-d865bca33e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:37.371014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 672/1327] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:37.407237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:37.408900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:37.515201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62f3e9-c2f5-4d76-a163-cb0e86405821 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:37.593025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:37.593758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:37.594048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:37.594331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:37.595995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:37.613503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:37.710491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:37.711980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad97d4-34e0-499b-8876-6e8142c98414 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:37.730957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 656/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:39.812583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:39.813139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:39.813139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:39.814492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 673/1329] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:39.819919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7ca9-a7a5-41ef-b975-e51eb348e501 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:39.820361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7ca9-a7a5-41ef-b975-e51eb348e501 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:39.820494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e7ca9-a7a5-41ef-b975-e51eb348e501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:39.890086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-670e7ca9-a7a5-41ef-b975-e51eb348e501 None None] MFA Rules not processed for user `a5a53c91bfd74877b8a19925a7e8da30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:39.891275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-670e7ca9-a7a5-41ef-b975-e51eb348e501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:39.958020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:39.994539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 657/1330] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:40.032033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:40.033561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:40.139200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de61292-dd3b-40fd-aed8-36dec479f477 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:40.159889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:40.218014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:40.218671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:40.218959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:40.219271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:40.220909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:40.328019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:40.329465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dbcbd-f35d-4d62-9cb7-3dbda008ff75 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:40.353141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 674/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:43.059966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.060527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.060527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.061893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 658/1332] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:43.067148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660e71c-2d1a-4724-ae4d-8418addf12f3 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.067547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660e71c-2d1a-4724-ae4d-8418addf12f3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.067710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660e71c-2d1a-4724-ae4d-8418addf12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.130354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:43.134176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b660e71c-2d1a-4724-ae4d-8418addf12f3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:43.135418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b660e71c-2d1a-4724-ae4d-8418addf12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:43.231333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 675/1333] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:43.268641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:43.270484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:43.374731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fe785-69cc-48f8-8701-593a5e156afb None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:43.446779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:43.447446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.447732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.448009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.449655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:43.476920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:43.557977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:43.559454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497343c4-23d7-4296-84fd-abfc31689da4 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:43.577354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 659/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:45.640091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.640624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.640752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.642025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 676/1335] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:45.647429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a5d6e-82a5-4928-bf34-8d064487ebe6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.647856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a5d6e-82a5-4928-bf34-8d064487ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.647996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a5d6e-82a5-4928-bf34-8d064487ebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.713155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3f1a5d6e-82a5-4928-bf34-8d064487ebe6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:45.714424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3f1a5d6e-82a5-4928-bf34-8d064487ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:45.789162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:45.810405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 660/1336] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:45.851932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:45.853550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:45.956111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69759-a00b-4233-86b6-95e454de8bd7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:45.990366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:46.039905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:46.040560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:46.040844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:46.041146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:46.042781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:46.149809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:46.151324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56e3fa-6060-4be8-b089-3165db500cd1 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:46.170120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 677/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:48.761642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.762152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.762257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.763626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 661/1338] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:48.769388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367d9b-ea4a-49e3-9267-70e48636c3a8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.769861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367d9b-ea4a-49e3-9267-70e48636c3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.769959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367d9b-ea4a-49e3-9267-70e48636c3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.833374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-09367d9b-ea4a-49e3-9267-70e48636c3a8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:48.834811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-09367d9b-ea4a-49e3-9267-70e48636c3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:48.842909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:48.930094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 678/1339] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:48.966066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.967648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:49.069837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e81a73-b848-49b6-9d9d-d518b1cf8b81 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:49.141371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:49.142015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:49.142303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:49.142590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:49.144249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:49.180623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:49.253988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:49.255475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605fe2-7bd3-4d19-8984-c4e91942c237 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:49.274141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 662/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:53.568155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:53.569017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:53.569313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:53.570932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 679/1341] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:53.576231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6349-ec3a-4424-aeb9-0f737d23e15e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:53.576671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6349-ec3a-4424-aeb9-0f737d23e15e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:53.576867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6349-ec3a-4424-aeb9-0f737d23e15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:53.647924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-eeff6349-ec3a-4424-aeb9-0f737d23e15e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:53.649136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-eeff6349-ec3a-4424-aeb9-0f737d23e15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:53.744730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:53.756205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 663/1342] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:53.795529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:53.797143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:53.915074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294c3d4-6a0c-4505-8805-74b31b3dcbd7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:53.957868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:54.005377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:54.006566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:54.007324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:54.007790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:54.010666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:54.143282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:54.144758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d76a8-b91e-4d82-b7a6-ca0a0f95d8b6 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:54.171372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 680/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:57.247852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:57.248399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:57.248399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:57.249812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 664/1344] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:57.255336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1687d6-0936-4efa-9754-949d6f2d8e6f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:57.256470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1687d6-0936-4efa-9754-949d6f2d8e6f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:57.256621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1687d6-0936-4efa-9754-949d6f2d8e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:57.321559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9d1687d6-0936-4efa-9754-949d6f2d8e6f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:57.323235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9d1687d6-0936-4efa-9754-949d6f2d8e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:57.341836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:57.422266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 681/1345] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:08:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:57.459875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:57.460993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:57.565985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973f3a-79db-4846-a8d2-353a8efe54de None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:57.644408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:57.645064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:57.645354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:57.645629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:57.647306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:57.690840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:57.758186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:57.759684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f7928-75c0-4f55-a4dd-114dd9531951 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:57.778860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 665/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:02.025178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.025710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.025810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.027340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 682/1347] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:02.032658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b821-4bb9-4b32-a898-22bfde822dd1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.033150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b821-4bb9-4b32-a898-22bfde822dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.033257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b821-4bb9-4b32-a898-22bfde822dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.103290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-13a8b821-4bb9-4b32-a898-22bfde822dd1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:02.104533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-13a8b821-4bb9-4b32-a898-22bfde822dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.207707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:02.210768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 666/1348] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:02.250299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.251918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.362512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346726f4-1ec6-45b6-9cb1-06d31bf6931b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.410030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:02.438804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.439778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.440202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.440760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.442385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.548939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.550412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961008-6f77-44d5-b0c5-1b62c604e699 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.568031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 683/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:07.528068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcf39-36e3-428a-87a1-2390192cdb51 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.528630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcf39-36e3-428a-87a1-2390192cdb51 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.528630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcf39-36e3-428a-87a1-2390192cdb51 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.535153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 667/1350] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:07.551143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd77f5e-885c-4fbb-bd19-d9b5eb3bb509 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.551143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd77f5e-885c-4fbb-bd19-d9b5eb3bb509 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.551143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd77f5e-885c-4fbb-bd19-d9b5eb3bb509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.632471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3bd77f5e-885c-4fbb-bd19-d9b5eb3bb509 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:07.633675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3bd77f5e-885c-4fbb-bd19-d9b5eb3bb509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:07.653332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:07.766290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 684/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:09:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:11.877847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5da48-f972-4ee1-be58-dc77952058a5 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.878448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5da48-f972-4ee1-be58-dc77952058a5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.878448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5da48-f972-4ee1-be58-dc77952058a5 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.880279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 668/1352] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:11.885219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a1f8-bdd5-44bd-88b7-eb62b778ecb6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.886021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a1f8-bdd5-44bd-88b7-eb62b778ecb6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.886522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a1f8-bdd5-44bd-88b7-eb62b778ecb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.971621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fa57a1f8-bdd5-44bd-88b7-eb62b778ecb6 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:11.973174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fa57a1f8-bdd5-44bd-88b7-eb62b778ecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.054369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:12.100433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 685/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:09:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:13.529017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9cb3-5418-4088-8eaf-81cad7a5119f None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.529559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9cb3-5418-4088-8eaf-81cad7a5119f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.529559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9cb3-5418-4088-8eaf-81cad7a5119f None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.530956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 669/1354] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:13.537011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4012-ed21-4cc5-9c6d-853f59c5a88b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.537477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4012-ed21-4cc5-9c6d-853f59c5a88b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.537621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4012-ed21-4cc5-9c6d-853f59c5a88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.612772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-321e4012-ed21-4cc5-9c6d-853f59c5a88b None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:13.614177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-321e4012-ed21-4cc5-9c6d-853f59c5a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.726156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 686/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:09:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:17.204321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568ad46-5ab3-424e-a23f-3e80fce4d93f None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.205168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568ad46-5ab3-424e-a23f-3e80fce4d93f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.205629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568ad46-5ab3-424e-a23f-3e80fce4d93f None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.211055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 670/1356] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:17.213132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ee23a-b0b0-44d9-b62a-78086420e8d2 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.215330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ee23a-b0b0-44d9-b62a-78086420e8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.215879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ee23a-b0b0-44d9-b62a-78086420e8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.253966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:17.290732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7d4ee23a-b0b0-44d9-b62a-78086420e8d2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:17.292459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7d4ee23a-b0b0-44d9-b62a-78086420e8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.399451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 687/1357] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:17.454556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.455848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.456619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.460077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 671/1358] 10.222.0.59 () {54 vars in 961 bytes} [Thu Jul 14 11:09:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:17.469466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36de2-57e0-41c6-9364-94e35e38a7ce None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.470062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36de2-57e0-41c6-9364-94e35e38a7ce None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.470639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f36de2-57e0-41c6-9364-94e35e38a7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.554578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c2f36de2-57e0-41c6-9364-94e35e38a7ce None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:17.555960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c2f36de2-57e0-41c6-9364-94e35e38a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.609330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:17.687249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 688/1359] 10.222.0.59 () {58 vars in 1058 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:17.695772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.697706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.872853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1695d-274f-44d3-b352-7646edc4eec9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.951518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:17.993452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.994102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.994391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.994664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.996516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:18.097139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.099595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01665852-9850-43ca-ae3a-f327fcfce363 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.119361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 672/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:20.386792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:20.387327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:20.387408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:20.388893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 689/1361] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:20.393862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72cf0-fb0e-4f68-af85-04e12e90fe1e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:20.394375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72cf0-fb0e-4f68-af85-04e12e90fe1e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:20.394461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72cf0-fb0e-4f68-af85-04e12e90fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:20.461765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5dd72cf0-fb0e-4f68-af85-04e12e90fe1e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:20.462960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5dd72cf0-fb0e-4f68-af85-04e12e90fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:20.501592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:20.567615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 673/1362] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:20.611698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:20.613425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:20.781575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651222-bc43-4959-b65d-f004e4bf70bc None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:20.792490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:20.914688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:20.915422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:20.915709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:20.915986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:20.917841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:21.014818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:21.016466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce9238-554a-4383-8bc2-66b7784dbf3d service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:21.036354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 690/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:23.556891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f56ae-f89c-4ff2-bb17-b8c2706125e4 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.557449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f56ae-f89c-4ff2-bb17-b8c2706125e4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.557449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f56ae-f89c-4ff2-bb17-b8c2706125e4 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.558818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 674/1364] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:23.564257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151251e3-095f-4212-a0b8-26addd32e031 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.564663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151251e3-095f-4212-a0b8-26addd32e031 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.564864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151251e3-095f-4212-a0b8-26addd32e031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.580338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:23.640012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-151251e3-095f-4212-a0b8-26addd32e031 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:23.641521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-151251e3-095f-4212-a0b8-26addd32e031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:23.746165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 691/1365] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:23.794236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.794759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.795402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.796893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 675/1366] 10.222.0.59 () {54 vars in 961 bytes} [Thu Jul 14 11:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:23.803985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167802f2-677f-48be-b8d3-4e254a097111 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.804279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167802f2-677f-48be-b8d3-4e254a097111 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.804555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167802f2-677f-48be-b8d3-4e254a097111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.885288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-167802f2-677f-48be-b8d3-4e254a097111 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:23.886546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-167802f2-677f-48be-b8d3-4e254a097111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:23.922598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:24.015540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 692/1367] 10.222.0.59 () {58 vars in 1058 bytes} [Thu Jul 14 11:09:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:24.025245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.027264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:24.197402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b409ff3-c1e1-4782-9d00-e2c94b04ece4 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.220862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:24.322166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.323099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.323417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.323701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.325362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:24.420388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.422011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d9b7-4f34-4e9d-96d4-a8337de1b714 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.443242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 676/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:25.395998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc991ad-69f7-4c47-bd82-e729b5cd139f None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.396562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc991ad-69f7-4c47-bd82-e729b5cd139f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.396562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc991ad-69f7-4c47-bd82-e729b5cd139f None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.398142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 693/1369] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:25.403301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc56e9b-f826-4307-ae03-9ec1fd6a6e9b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.403751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc56e9b-f826-4307-ae03-9ec1fd6a6e9b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.404052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc56e9b-f826-4307-ae03-9ec1fd6a6e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.438175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:25.473785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3dc56e9b-f826-4307-ae03-9ec1fd6a6e9b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:25.475173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3dc56e9b-f826-4307-ae03-9ec1fd6a6e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:25.577689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 677/1370] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:25.926316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.926609 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.926894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.928463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 694/1371] 10.222.0.59 () {54 vars in 963 bytes} [Thu Jul 14 11:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:25.933444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d476d2-b196-496f-892f-f034d6238d6e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.933741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d476d2-b196-496f-892f-f034d6238d6e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.934022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d476d2-b196-496f-892f-f034d6238d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.015828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-59d476d2-b196-496f-892f-f034d6238d6e None None] MFA Rules not processed for user `c86f8e914f3f4312a60791af443b2f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:26.017061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-59d476d2-b196-496f-892f-f034d6238d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.069759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:26.100402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 678/1372] 10.222.0.59 () {58 vars in 1060 bytes} [Thu Jul 14 11:09:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:26.108197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:26.109976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.239704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da49756-84bc-4dc2-b74f-5e3413d625a4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:26.286787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:26.320527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:26.321272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:26.321627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:26.321975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.323741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.570916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:26.572945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc653d1-67ff-4670-b82c-1338b1f8414f service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:26.599549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 695/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:26.720322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:26.720746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:26.720971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.722839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 679/1374] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:26.728134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f12c1-b766-4331-b543-707d5df6be2b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:26.728459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f12c1-b766-4331-b543-707d5df6be2b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:26.728942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f12c1-b766-4331-b543-707d5df6be2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.804310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a26f12c1-b766-4331-b543-707d5df6be2b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:26.805891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a26f12c1-b766-4331-b543-707d5df6be2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.830868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:26.926434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 696/1375] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:26.970676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:26.972328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:27.121054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe40d6-ffec-4d24-88ec-f8b7b70d8dcc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:27.175030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:27.240823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:27.241486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:27.241777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:27.242060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:27.243755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:27.334795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:27.336461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ce2ef-fd9c-416d-a80b-4f11b729c2c9 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:27.356130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 680/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:29.480283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.480754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.480883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.482221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 697/1377] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:29.487466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877b4a-5f89-4cee-95b9-2bb517dc4156 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.488033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877b4a-5f89-4cee-95b9-2bb517dc4156 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.488033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a877b4a-5f89-4cee-95b9-2bb517dc4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.555760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6a877b4a-5f89-4cee-95b9-2bb517dc4156 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:29.556971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a877b4a-5f89-4cee-95b9-2bb517dc4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:29.563522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:29.664624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 681/1378] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:29.712712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:29.714249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:29.867690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f865e73-60b7-4651-a3d5-74f9f5d4387b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:29.871512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:29.984464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:29.985118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.985449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.985777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.987438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:30.083840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:30.085185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae89274-b77b-420c-a966-3ea95e703621 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:30.106316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 698/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:32.335400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.336062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.336062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.338068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 682/1380] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:32.344067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48438fcc-153b-4fc6-811f-60b33707223f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.344624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48438fcc-153b-4fc6-811f-60b33707223f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.344800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48438fcc-153b-4fc6-811f-60b33707223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.356518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:32.414696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-48438fcc-153b-4fc6-811f-60b33707223f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:32.416245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-48438fcc-153b-4fc6-811f-60b33707223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.519258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 699/1381] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:32.561733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.563307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.714712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:32.725539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d549-791f-4f80-be08-513d1d936d8b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.847937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.848762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.849147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.849530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.851346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.947364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.949144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19e69-d15c-42fd-855f-f2d063c3c14f service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.960786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:32.969875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 683/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:35.212588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.213149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.213149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.214539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 700/1383] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:35.219799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10af08-cd99-4464-94b9-5521ac71b20e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.220225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10af08-cd99-4464-94b9-5521ac71b20e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.220381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10af08-cd99-4464-94b9-5521ac71b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.302839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3d10af08-cd99-4464-94b9-5521ac71b20e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:35.304480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3d10af08-cd99-4464-94b9-5521ac71b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.418757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 684/1384] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:35.461690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.463246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.549344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:35.612100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda9522-b4d4-47ea-8af1-4ee68a612e44 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.730882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.731579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.731865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.732148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.733788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.785008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:35.827227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.829105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe01d3-e3b8-4835-ad37-9683fabdd8ff service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.847898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 701/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:37.883177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:37.883829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:37.883829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:37.885095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 685/1386] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:09:37.890527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0dfab-a9ad-41af-a6d5-4ae747e13bcf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:37.890954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0dfab-a9ad-41af-a6d5-4ae747e13bcf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:37.891083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0dfab-a9ad-41af-a6d5-4ae747e13bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:37.957805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-94e0dfab-a9ad-41af-a6d5-4ae747e13bcf None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:37.959045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-94e0dfab-a9ad-41af-a6d5-4ae747e13bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:38.059663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 702/1387] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:38.104003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:38.105986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:38.204214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:38.282716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dec4c-6250-4d52-8fa8-a66c547476b3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:38.408007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:38.408535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:38.408821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:38.409094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:38.410815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:38.450813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:38.508766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:38.510398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7e34-3cd4-4588-93c4-3ed932a312dd service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:38.529475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 686/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:40.734705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:40.735326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:40.735326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:40.736682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 703/1389] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:40.742046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a797-c6fc-4d6f-bd64-ff70b4ea6311 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:40.742453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a797-c6fc-4d6f-bd64-ff70b4ea6311 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:40.742619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a797-c6fc-4d6f-bd64-ff70b4ea6311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:40.810415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ec77a797-c6fc-4d6f-bd64-ff70b4ea6311 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:40.811689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ec77a797-c6fc-4d6f-bd64-ff70b4ea6311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:40.913209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 687/1390] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:40.958119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:40.959763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:40.991272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:41.107020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e8f1-ec08-4f72-b5b0-695db42698ee None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:41.216293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:41.228729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:41.229423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:41.229707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:41.229989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:41.231655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:41.326824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:41.328377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e069-5d78-4ffe-9201-32e5a9feb702 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:41.346510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 704/1391] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:42.197548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b616b3-617a-40c4-a087-a82c5ec724e2 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:42.198678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b616b3-617a-40c4-a087-a82c5ec724e2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:42.198678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b616b3-617a-40c4-a087-a82c5ec724e2 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:42.199628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 688/1392] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:42.204707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1189f5-d14c-4276-bc20-69fd2811db18 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:42.205131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1189f5-d14c-4276-bc20-69fd2811db18 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:42.205265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1189f5-d14c-4276-bc20-69fd2811db18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:42.270057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1f1189f5-d14c-4276-bc20-69fd2811db18 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:42.271299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1f1189f5-d14c-4276-bc20-69fd2811db18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:42.328884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:42.367958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 705/1393] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:42.423533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:42.423824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:42.424102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:42.425465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 689/1394] 10.222.0.59 () {54 vars in 962 bytes} [Thu Jul 14 11:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:42.431344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed50c-f4ad-4dd2-be78-1d3155eba2da None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:42.431639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed50c-f4ad-4dd2-be78-1d3155eba2da None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:42.431920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed50c-f4ad-4dd2-be78-1d3155eba2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:42.505420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a4eed50c-f4ad-4dd2-be78-1d3155eba2da None None] MFA Rules not processed for user `bbfd6e827b9e49949cd6543edc89bbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:42.506611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a4eed50c-f4ad-4dd2-be78-1d3155eba2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:42.589081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 706/1395] 10.222.0.59 () {58 vars in 1059 bytes} [Thu Jul 14 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:42.599199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:42.601341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:42.620972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:42.721256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702390dc-d329-494d-ac87-4935c7bd9936 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:42.799866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:42.800516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:42.800801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:42.801707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:42.803508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:42.876302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:42.918461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:42.919931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12b8bf-b071-4f15-a691-c84ee3a744d7 service glance] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:42.939338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 690/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:43.595525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:43.596088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:43.596088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:43.597668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 707/1397] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:43.604261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64281af-6bbf-4a49-9d5a-01027b46795b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:43.606436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64281af-6bbf-4a49-9d5a-01027b46795b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:43.606436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64281af-6bbf-4a49-9d5a-01027b46795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:43.678543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c64281af-6bbf-4a49-9d5a-01027b46795b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:43.679829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c64281af-6bbf-4a49-9d5a-01027b46795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:43.776695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 691/1398] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:43.823089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:43.823750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7dcc-67db-4d1a-9606-79ab9f3dfa53 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:43.824055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7dcc-67db-4d1a-9606-79ab9f3dfa53 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:43.824357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b7dcc-67db-4d1a-9606-79ab9f3dfa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:43.824886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:43.855278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:43.890546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:43.895582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2c9b7dcc-67db-4d1a-9606-79ab9f3dfa53 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:43.898797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2c9b7dcc-67db-4d1a-9606-79ab9f3dfa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:43.977051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f447ea-e8a8-4ec3-bdeb-e5231ac39325 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:43.986528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 692/1399] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:09:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:44.030740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055ea03-6c78-4f34-b168-11e49d0bae20 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:44.031019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055ea03-6c78-4f34-b168-11e49d0bae20 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:44.031290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055ea03-6c78-4f34-b168-11e49d0bae20 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:44.032715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 693/1400] 10.222.0.59 () {54 vars in 942 bytes} [Thu Jul 14 11:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:09:44.038801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d64cd-f524-49b7-811a-121941d15f7d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:44.039210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d64cd-f524-49b7-811a-121941d15f7d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:44.039498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d64cd-f524-49b7-811a-121941d15f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:44.095265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:44.096071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:44.096355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:44.096634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:44.098493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:44.115742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-487d64cd-f524-49b7-811a-121941d15f7d None None] MFA Rules not processed for user `7e631e04eeba421885436f9f2a1330b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:44.116959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-487d64cd-f524-49b7-811a-121941d15f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:44.120627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:44.172739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:44.198579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:44.200230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:44.219650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 708/1401] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:44.232058 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 694/1402] 10.222.0.59 () {58 vars in 1039 bytes} [Thu Jul 14 11:09:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:44.242329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:44.243617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:44.330877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55bf21-1f19-44d8-b308-188c94953a27 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:44.390478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:44.445599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:44.446268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:44.446550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:44.446835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:44.448610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:44.540372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:44.541970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3892-320b-4c39-a462-d4678aae1c7d service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:44.550183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 709/1403] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:09:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:09:46.391801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:46.392359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:46.392359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:46.393747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 695/1404] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:46.399791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60344-0000-4d62-93a3-8e58dabe8c0d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:46.400218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60344-0000-4d62-93a3-8e58dabe8c0d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:46.400413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60344-0000-4d62-93a3-8e58dabe8c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:46.469583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-82b60344-0000-4d62-93a3-8e58dabe8c0d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:46.470715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-82b60344-0000-4d62-93a3-8e58dabe8c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:46.528902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:46.601889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 710/1405] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:46.649224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:46.650753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:46.839275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e076a4-7649-4db4-96bc-4c1ecb872020 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:46.875872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:46.988780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:46.990806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:46.991093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:46.991399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:46.993136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:47.106257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:47.107960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9841bf0-1332-4179-b378-f1f51d7f824d service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:47.130116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 696/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:50.015554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82f87-be93-4123-a637-bcae862a611a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:50.016573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82f87-be93-4123-a637-bcae862a611a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:50.016573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82f87-be93-4123-a637-bcae862a611a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:50.017814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 711/1407] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:50.022967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a44f-a258-47f3-b7a8-3ad7c6290022 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:50.023446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a44f-a258-47f3-b7a8-3ad7c6290022 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:50.023567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a44f-a258-47f3-b7a8-3ad7c6290022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:50.066243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:50.092815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ea78a44f-a258-47f3-b7a8-3ad7c6290022 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:50.094125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ea78a44f-a258-47f3-b7a8-3ad7c6290022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:50.214653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 697/1408] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:50.259965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82f87-be93-4123-a637-bcae862a611a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:50.260463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-aab82f87-be93-4123-a637-bcae862a611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:50.425833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82f87-be93-4123-a637-bcae862a611a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:50.458232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:50.553375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:50.553940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:50.554223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:50.554498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:50.556186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:50.650264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:50.656604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7608b-dd04-42f1-918f-46f975e47702 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:50.678902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 712/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:00.212580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=51 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:00.675987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=86 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:08.153725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec8a5-1fb3-434a-8562-26171add02f7 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:08.154304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec8a5-1fb3-434a-8562-26171add02f7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:08.154304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec8a5-1fb3-434a-8562-26171add02f7 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:08.155700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 698/1410] 10.222.0.59 () {54 vars in 918 bytes} [Thu Jul 14 11:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:08.161855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cec28-64f0-4828-8b61-116876846513 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:08.162309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cec28-64f0-4828-8b61-116876846513 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:08.162407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cec28-64f0-4828-8b61-116876846513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:08.239033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0b5cec28-64f0-4828-8b61-116876846513 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:08.240277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0b5cec28-64f0-4828-8b61-116876846513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:08.345075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 713/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Thu Jul 14 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:08.360098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de905a15-6213-435c-9293-2ca3cfe5dd53 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:08.360386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de905a15-6213-435c-9293-2ca3cfe5dd53 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:08.360665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de905a15-6213-435c-9293-2ca3cfe5dd53 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:08.362199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 699/1412] 10.222.0.59 () {54 vars in 918 bytes} [Thu Jul 14 11:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:08.366882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78e6a-a2f6-496c-a4e8-8f5f125a4174 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:08.367194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78e6a-a2f6-496c-a4e8-8f5f125a4174 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:08.367504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c78e6a-a2f6-496c-a4e8-8f5f125a4174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:08.438576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:08.441782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-06c78e6a-a2f6-496c-a4e8-8f5f125a4174 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:08.442985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-06c78e6a-a2f6-496c-a4e8-8f5f125a4174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:08.557001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 714/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Thu Jul 14 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:11.077812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:11.078378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:11.078378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:11.079759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:11.085315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300bd2b-1abf-4ef9-a002-e8a98a099696 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:11.085709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300bd2b-1abf-4ef9-a002-e8a98a099696 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:11.085875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300bd2b-1abf-4ef9-a002-e8a98a099696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:11.150143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5300bd2b-1abf-4ef9-a002-e8a98a099696 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:11.151402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5300bd2b-1abf-4ef9-a002-e8a98a099696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:11.208350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:11.249104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:11.292136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:11.293675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:11.438405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222716f0-74f0-4ce4-9f5d-049cdf25bd0d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:11.511426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:11.551615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:11.552291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:11.552575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:11.552850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:11.554550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:11.642534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:11.644182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61755ec3-7397-466e-9c61-aea6371b8634 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:11.662348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 701/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:12.564787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:12.565449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:12.565449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:12.566838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:12.571743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a208-d4c5-422a-85d3-55b0225d01ec None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:12.572148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a208-d4c5-422a-85d3-55b0225d01ec None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:12.572294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a208-d4c5-422a-85d3-55b0225d01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:12.646312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2955a208-d4c5-422a-85d3-55b0225d01ec None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:12.647607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2955a208-d4c5-422a-85d3-55b0225d01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:12.675078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:12.744330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:12.896145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:12.897577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:13.041357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a1762-3fdd-4384-875e-10c02c67ed9c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:13.082418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:13.157402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:13.157979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:13.158890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:13.159227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:13.160920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:13.250233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:13.252399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026952bf-0208-40a4-828c-8a826b1ead99 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:13.271295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 717/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:14.162570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:14.163178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:14.163178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:14.164572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:14.169708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae51-7bbc-45a3-9dfe-980df8f49e31 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:14.170123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae51-7bbc-45a3-9dfe-980df8f49e31 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:14.171659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8eae51-7bbc-45a3-9dfe-980df8f49e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:14.225283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:14.247031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5a8eae51-7bbc-45a3-9dfe-980df8f49e31 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:14.248262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5a8eae51-7bbc-45a3-9dfe-980df8f49e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:14.343284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:14.497091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:14.498614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:14.649913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab0b21-0a2d-4093-a4ee-d39d98d8e38f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:14.662479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:14.762226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:14.762793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:14.763073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:14.763375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:14.765010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:14.852421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:14.854035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6788b0-3151-43e5-8764-73587e8e093d service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:14.873196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 704/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:15.742517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:15.743082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:15.743082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:15.744512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:15.749348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81eaf-d05d-4d36-9417-f6d337dd01a8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:15.749757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81eaf-d05d-4d36-9417-f6d337dd01a8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:15.749910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81eaf-d05d-4d36-9417-f6d337dd01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:15.768952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:15.824764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c3c81eaf-d05d-4d36-9417-f6d337dd01a8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:15.825979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c3c81eaf-d05d-4d36-9417-f6d337dd01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:15.922075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:16.076039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:16.077484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:16.186674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:16.222508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb967f-7747-41f4-8c60-e690d486e626 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:16.332523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:16.333094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:16.333376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:16.333650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:16.335321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:16.410042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:16.423234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:16.425810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f13ab-1111-4f74-b4ce-a0cc71df2fd9 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:16.443533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 720/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:17.312684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:17.313246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:17.313246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:17.314726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:17.319722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779e23f-a05e-4f23-99d0-4c022bc4b189 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:17.320104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779e23f-a05e-4f23-99d0-4c022bc4b189 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:17.320273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779e23f-a05e-4f23-99d0-4c022bc4b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:17.396129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7779e23f-a05e-4f23-99d0-4c022bc4b189 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:17.397325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7779e23f-a05e-4f23-99d0-4c022bc4b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:17.491364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:17.643740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:17.645267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:17.728868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:17.792169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aad66f-53c4-48a1-b624-e2f1ec69e842 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:18.026326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:18.026900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:18.027216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:18.027496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:18.029153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:18.077853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:18.120451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:18.122062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbfc28-7ee1-4d0e-a958-4824e69bbd44 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:18.140982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 707/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:18.999971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:19.000502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:19.000502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:19.002940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:19.008104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b28f4-e97e-4d2d-bdcc-b6ad07e17091 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:19.008496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b28f4-e97e-4d2d-bdcc-b6ad07e17091 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:19.008659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b28f4-e97e-4d2d-bdcc-b6ad07e17091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:19.081778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ca9b28f4-e97e-4d2d-bdcc-b6ad07e17091 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:19.082899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ca9b28f4-e97e-4d2d-bdcc-b6ad07e17091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:19.179899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:19.335263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:19.336678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:19.386026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:19.481826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb14664-4f0d-47ca-9ac3-f7f78f2d0b78 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:19.595850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:19.596402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:19.596687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:19.596962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:19.598582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:19.605399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:19.688323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:19.690623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5378f94-e398-4754-a5ab-7b1ae8ad4db1 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:19.709444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 723/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:20.580608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:20.581175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:20.581276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:20.582581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:20.587597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ba75-c1a6-4443-a387-fb49855da3f4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:20.588019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ba75-c1a6-4443-a387-fb49855da3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:20.588152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ba75-c1a6-4443-a387-fb49855da3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:20.662279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3052ba75-c1a6-4443-a387-fb49855da3f4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:20.663513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3052ba75-c1a6-4443-a387-fb49855da3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:20.732359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:20.760968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:20.919256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:20.920761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:20.939617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:21.065983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5905-b881-4d4c-93b2-dcddc1975003 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:21.152797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:21.179542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:21.180285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:21.180465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:21.181278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:21.183292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:21.273123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:21.274727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96448-8295-4bed-ab4b-966394170596 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:21.292668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 710/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:22.170948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.171506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.171600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.173810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:22.179394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daf78-4fe9-49c7-b613-f304159440e5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.179845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daf78-4fe9-49c7-b613-f304159440e5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.180413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daf78-4fe9-49c7-b613-f304159440e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.254479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0b9daf78-4fe9-49c7-b613-f304159440e5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:22.255703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0b9daf78-4fe9-49c7-b613-f304159440e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.295669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:22.352367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:22.507184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:22.508613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.652999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1a6f4-b4d9-4edd-8cc6-390ea877aa1a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:22.707728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:22.766188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:22.766772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.767320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.767612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.769254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.858461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:22.860087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa76166-eed0-45e8-9f98-ee69756e22d1 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:22.878830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 726/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:23.738808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.739399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.739399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.740784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:23.745618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090510a4-77ab-4ed2-8d38-5211a50810e7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.746041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090510a4-77ab-4ed2-8d38-5211a50810e7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.746179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090510a4-77ab-4ed2-8d38-5211a50810e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.821974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-090510a4-77ab-4ed2-8d38-5211a50810e7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:23.823218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-090510a4-77ab-4ed2-8d38-5211a50810e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:23.825199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:23.919160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:24.072432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:24.133752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.223863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de52cd7-d130-4eb5-9262-c49e97446f12 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:24.252567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:24.335875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:24.336520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.336800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.337092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.338702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.427211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:24.428814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a4ecf-2596-4bfc-882d-0557f3c04ff7 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:24.446566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 713/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:26.372544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:26.373118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:26.373118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:26.374477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:26.380540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde22e-26e7-462e-b703-fe8a2a9e4c97 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:26.380938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde22e-26e7-462e-b703-fe8a2a9e4c97 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:26.381112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde22e-26e7-462e-b703-fe8a2a9e4c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:26.419562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:26.445679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a8bde22e-26e7-462e-b703-fe8a2a9e4c97 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:26.446807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a8bde22e-26e7-462e-b703-fe8a2a9e4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:26.542707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:26.582058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:26.583612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:26.685794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d59f0c-e751-4afe-98eb-220ec706d550 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:26.717220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:26.759340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:26.759989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:26.760273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:26.760552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:26.762175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:26.872121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:26.873559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d500ed-e6f8-41fe-a399-6036bd459e87 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:26.893413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 729/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:29.082785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:29.083381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:29.083381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:29.084674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:29.090004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e428e1-c3f7-4c31-ac3e-f7e3e7f80eb4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:29.090411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e428e1-c3f7-4c31-ac3e-f7e3e7f80eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:29.090571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e428e1-c3f7-4c31-ac3e-f7e3e7f80eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:29.156106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-33e428e1-c3f7-4c31-ac3e-f7e3e7f80eb4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:29.157361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-33e428e1-c3f7-4c31-ac3e-f7e3e7f80eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:29.172457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:29.255723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:29.293309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:29.297464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:29.398798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d575e-be1b-4c5c-9508-29c03c0718f2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:29.410880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:29.472354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:29.473007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:29.473294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:29.473573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:29.475253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:29.583421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:29.584853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53670f50-ad28-413c-89ee-499044ebef02 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:29.603654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 716/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:32.019481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:32.020069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:32.020069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:32.021459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:32.026893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f026-4a3f-4361-b9a5-e74b4399b8d4 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:32.027365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f026-4a3f-4361-b9a5-e74b4399b8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:32.027486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f026-4a3f-4361-b9a5-e74b4399b8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:32.074090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:32.092012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8c15f026-4a3f-4361-b9a5-e74b4399b8d4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:32.093222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8c15f026-4a3f-4361-b9a5-e74b4399b8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:32.190424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:32.230979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:32.232571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:32.334105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8c86a-42e4-4180-9d42-76b620c4c93a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:32.373403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:32.406486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:32.407883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:32.408423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:32.408914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:32.410895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:32.517583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:32.519028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374656a-6789-4544-a3c3-a89624e28541 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:32.538056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 732/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:34.742272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:34.742867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:34.742867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:34.744225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:34.749364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6822a-48b6-44c2-95dc-c7b2f3ae9448 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:34.749769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6822a-48b6-44c2-95dc-c7b2f3ae9448 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:34.749929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6822a-48b6-44c2-95dc-c7b2f3ae9448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:34.815113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bdf6822a-48b6-44c2-95dc-c7b2f3ae9448 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:34.816350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bdf6822a-48b6-44c2-95dc-c7b2f3ae9448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:34.836559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:34.913274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:34.951678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:34.953207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:35.057556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b9d5a-e64c-43f9-a610-c5bc0b33e6a0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:35.080018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:35.133082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:35.133734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:35.134023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:35.134307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:35.136003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:35.245934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:35.247407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c47a0-2a41-4898-bb70-602397c4422c service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:35.266535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 719/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:37.366597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:37.367199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:37.367199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:37.368540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:37.373786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95b5f-fb27-4cbb-ba09-29026d889412 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:37.374192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95b5f-fb27-4cbb-ba09-29026d889412 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:37.374352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95b5f-fb27-4cbb-ba09-29026d889412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:37.436718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:37.440331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3fd95b5f-fb27-4cbb-ba09-29026d889412 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:37.442406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3fd95b5f-fb27-4cbb-ba09-29026d889412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:37.537507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:10:37.575876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:37.577485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:37.680082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192ee58-98d3-4d21-99e0-862075dc68af None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:37.728420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:37.753956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:37.754591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:37.754874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:37.755175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:37.757425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:37.864224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:37.865670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a8b16-bbcb-436e-a6e3-2bc9c947b08a service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:37.884340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 735/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:39.979820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:39.980388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:39.980388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:39.981738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:39.987700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56f9f-9fbc-4b94-ad40-4b3ae492a404 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:39.988151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56f9f-9fbc-4b94-ad40-4b3ae492a404 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:39.988260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56f9f-9fbc-4b94-ad40-4b3ae492a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:40.051685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-95c56f9f-9fbc-4b94-ad40-4b3ae492a404 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:40.052914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-95c56f9f-9fbc-4b94-ad40-4b3ae492a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:40.081969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:40.150402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:40.190782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:40.192391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:40.295782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb221d7-a31a-4dd3-8bef-003eaa0c719c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:40.323467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:40.367331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:40.368184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:40.368695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:40.369309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:40.370949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:40.477826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:40.479282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb4d3a-59d0-4992-899f-a580ca6e1774 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:40.497984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 722/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:42.695746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:42.696296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:42.696296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:42.697712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:42.703990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d991-4b1a-4c0d-a4c9-c6f327c39d98 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:42.704538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d991-4b1a-4c0d-a4c9-c6f327c39d98 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:42.704666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d991-4b1a-4c0d-a4c9-c6f327c39d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:42.768834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1378d991-4b1a-4c0d-a4c9-c6f327c39d98 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:42.770036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1378d991-4b1a-4c0d-a4c9-c6f327c39d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:42.778163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:42.869343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:42.908596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:42.910155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:43.012692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc98f2-6001-4ed7-bd3f-6ab5f6a1d284 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:43.068376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:43.087171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:43.087824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:43.088106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:43.088380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:43.090006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:43.199698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:43.201151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1acb-a274-4eb2-91e4-27c4f9913ca9 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:43.220526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 738/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:45.300545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:45.301102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:45.301102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:45.302497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:45.307742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1ca5-e42f-452f-bfbd-62da177d65c5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:45.308136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1ca5-e42f-452f-bfbd-62da177d65c5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:45.308298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1ca5-e42f-452f-bfbd-62da177d65c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:45.372932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7d5e1ca5-e42f-452f-bfbd-62da177d65c5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:45.374149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7d5e1ca5-e42f-452f-bfbd-62da177d65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:45.411720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:45.468837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:45.506915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:45.508529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:45.611561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefecaf0-d1ca-4382-927e-ebc545fffa75 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:45.648382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:45.684998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:45.685646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:45.685933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:45.686209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:45.687920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:45.796400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:45.798233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941fe97-e67a-45d8-944b-cb08a38ae6c9 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:45.817663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 725/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:47.969389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:47.969946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:47.969946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:47.971361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:47.976608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57f2b-9dfd-43a3-8f1d-f3494a607781 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:47.977058 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57f2b-9dfd-43a3-8f1d-f3494a607781 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:47.977154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57f2b-9dfd-43a3-8f1d-f3494a607781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:48.043278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-32f57f2b-9dfd-43a3-8f1d-f3494a607781 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:48.044420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-32f57f2b-9dfd-43a3-8f1d-f3494a607781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:48.064107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:48.142665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:48.180029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:48.181592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:48.295057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d57bf0-7fe3-4810-836e-c567cd219c51 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:48.362246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:48.368716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:48.369371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:48.369661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:48.369940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:48.371694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:48.478834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:48.480308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65948606-5ed5-4557-9543-dbd5d89834bf service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:48.498561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 741/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:50.556332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:50.556871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:50.556871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:50.558283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:10:50.564273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d5d2-ef87-4de6-9cc7-03045c1a47f0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:50.564685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d5d2-ef87-4de6-9cc7-03045c1a47f0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:50.564837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d5d2-ef87-4de6-9cc7-03045c1a47f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:50.629131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-09c4d5d2-ef87-4de6-9cc7-03045c1a47f0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:50.630333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-09c4d5d2-ef87-4de6-9cc7-03045c1a47f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:50.677718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:50.725575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:50.762870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:50.764498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:50.868649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb784ef3-81fc-4c50-a29b-84c59c6f84e1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:50.919067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:50.945380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:50.946039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:50.946324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:50.946600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:50.948341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:51.057511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:51.058996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39561114-70de-4069-b195-ba129a9d385e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:51.077946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 728/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:00.736787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=22 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:01.075476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=66 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:11.790269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037b0f-d4db-4cf4-94ce-cee34202a076 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:11.790846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037b0f-d4db-4cf4-94ce-cee34202a076 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:11.790846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037b0f-d4db-4cf4-94ce-cee34202a076 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:11.792228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 743/1471] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:11.798176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acc9f4-f788-4d63-9f0d-af85c2819c54 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:11.798618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acc9f4-f788-4d63-9f0d-af85c2819c54 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:11.798767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acc9f4-f788-4d63-9f0d-af85c2819c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:11.863353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d3acc9f4-f788-4d63-9f0d-af85c2819c54 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:11.864563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d3acc9f4-f788-4d63-9f0d-af85c2819c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:11.959780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 729/1472] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:12.046762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:12.047045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:12.047361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:12.048721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 744/1473] 10.222.0.59 () {54 vars in 962 bytes} [Thu Jul 14 11:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:12.053623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45e3fd-8928-4265-b7b7-a3cbb97724a9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:12.053915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45e3fd-8928-4265-b7b7-a3cbb97724a9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:12.054194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45e3fd-8928-4265-b7b7-a3cbb97724a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:12.127677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fc45e3fd-8928-4265-b7b7-a3cbb97724a9 None None] MFA Rules not processed for user `bbfd6e827b9e49949cd6543edc89bbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:12.128876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fc45e3fd-8928-4265-b7b7-a3cbb97724a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:12.149304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:12.204657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 730/1474] 10.222.0.59 () {58 vars in 1059 bytes} [Thu Jul 14 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:12.212112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:12.213605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:12.327289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81967e0d-38dd-4b67-9cc8-d8505275471d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:12.401062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:12.401704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:12.401986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:12.402261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:12.404897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:12.458216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:12.514003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:12.515454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6405cf8-8ab5-45b0-b9c1-1305b18b866e service glance] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:12.534859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 745/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:14.688683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:14.689227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:14.689326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:14.690696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 731/1476] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:14.696096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70b8d-14b4-4fc2-9c72-1b7063e3b0fd None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:14.696527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70b8d-14b4-4fc2-9c72-1b7063e3b0fd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:14.696668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70b8d-14b4-4fc2-9c72-1b7063e3b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:14.760547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d7a70b8d-14b4-4fc2-9c72-1b7063e3b0fd None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:14.761755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d7a70b8d-14b4-4fc2-9c72-1b7063e3b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:14.858527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 746/1477] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:14.882293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:14.883887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:15.006667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b50a11-c0de-42fd-a1f6-275ed2ffaadb None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:15.057112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:15.101477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:15.102139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:15.102423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:15.102698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:15.103464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:15.105208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:11:15.105208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:11:15.105208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:11:15.106347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a63c8-dc7f-48c5-8ebc-ad8e07b10229 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:15.126410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 732/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Thu Jul 14 11:11:14 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:11:17.225371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:17.225942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:17.225942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:17.227318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 747/1479] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:17.232719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8501b-1676-4e44-8565-a6ca9ed75ec4 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:17.233171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8501b-1676-4e44-8565-a6ca9ed75ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:17.233282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8501b-1676-4e44-8565-a6ca9ed75ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:17.298106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-60e8501b-1676-4e44-8565-a6ca9ed75ec4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:17.299356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-60e8501b-1676-4e44-8565-a6ca9ed75ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:17.395619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 733/1480] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:17.438468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:17.439992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:17.449684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:17.584639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ec6dd-dca1-4359-ac49-69af6904fa64 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:17.664481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:17.697889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:17.698478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:17.698765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:17.699068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:17.701364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:17.792591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:17.794373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e22664-6ee8-4043-a4aa-cac4d13de382 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:17.813043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 748/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:20.029675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:20.029675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:20.030146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:20.031364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 734/1482] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:20.036882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994599d7-edab-449d-ab93-3f5edcc4c40e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:20.037318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994599d7-edab-449d-ab93-3f5edcc4c40e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:20.037513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994599d7-edab-449d-ab93-3f5edcc4c40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:20.102085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-994599d7-edab-449d-ab93-3f5edcc4c40e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:20.103339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-994599d7-edab-449d-ab93-3f5edcc4c40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:20.148631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:20.199764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 749/1483] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:20.237054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:20.238623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:20.263773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:20.343445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963b0b9-f19c-45cd-b0cb-5e6f062d0623 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:20.415228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:20.415873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:20.416156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:20.416431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:20.418063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:20.496099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:20.532671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:20.534116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dfc27-576a-4794-83de-bfa1bbd1b7ae service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:20.552644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 735/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:22.647621 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:22.648175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:22.648280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:22.649576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 750/1485] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:22.654967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d5696-6fe4-451f-a102-1f0be29e59b8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:22.655410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d5696-6fe4-451f-a102-1f0be29e59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:22.655575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d5696-6fe4-451f-a102-1f0be29e59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:22.721930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5b9d5696-6fe4-451f-a102-1f0be29e59b8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:22.723278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5b9d5696-6fe4-451f-a102-1f0be29e59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:22.821244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 736/1486] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:22.858947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:22.860508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:22.962440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7123da7-d5b1-41f3-b43a-9be16fc899fa None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:23.036179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:23.037944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:23.038748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:23.039036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:23.039338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:23.041016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:23.152521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:23.154480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa570-509e-44af-8f2f-5df8a7133138 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:23.174265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 751/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:25.368448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:25.368984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:25.368984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:25.370389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 737/1488] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:25.376454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908085-27f0-47a1-a2a3-efb891cc706a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:25.376878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908085-27f0-47a1-a2a3-efb891cc706a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:25.377005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e908085-27f0-47a1-a2a3-efb891cc706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:25.440899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8e908085-27f0-47a1-a2a3-efb891cc706a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:25.442012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8e908085-27f0-47a1-a2a3-efb891cc706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:25.494266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:25.538017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 752/1489] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:25.578666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:25.580243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:25.616047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:25.683351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db924f-eb38-4576-a7fb-10d8c910fe93 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:25.757884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:25.758543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:25.758832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:25.759110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:25.760842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:25.840915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:25.870752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:25.872870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df3929-a43c-442b-97ff-f5275c90460f service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:25.892083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 738/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:28.421261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:28.421796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:28.421925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:28.423269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 753/1491] 10.222.0.59 () {54 vars in 926 bytes} [Thu Jul 14 11:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:28.428384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec9d1-14c4-426f-a8d1-fa108d2ce213 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:28.428783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec9d1-14c4-426f-a8d1-fa108d2ce213 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:28.428998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ec9d1-14c4-426f-a8d1-fa108d2ce213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:28.493676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dc6ec9d1-14c4-426f-a8d1-fa108d2ce213 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:28.494891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dc6ec9d1-14c4-426f-a8d1-fa108d2ce213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:28.589645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 739/1492] 10.222.0.59 () {58 vars in 1023 bytes} [Thu Jul 14 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:28.629441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:28.631000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:28.742603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f0bda-b474-4637-8084-6bb5ebb91518 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:28.816378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:28.817032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:28.817317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:28.817594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:28.819254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:28.829082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:28.929321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:28.930756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e4b3d-52e4-45ff-89af-4a8a848f2ff9 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:28.948729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 754/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:38.587866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=82 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:38.946494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=50 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:04.122351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:04.122950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:04.122950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:04.201949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:04.203196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:04.300510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 740/1494] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:13:04.310452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:04.310958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:04.311068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:04.387314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:04.388434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:04.484060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 755/1495] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:13:04.488784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:04.490002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:04.502339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541009e8-3fc7-43fa-9cbe-48cecbe3b84b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:04.562813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:04.596028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:04.596686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:04.596964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:04.597237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:04.597951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:13:04.598257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:04.599296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:04.599296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:13:04.599296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:13:04.600795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:04.611893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 741/1496] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:13:04.621074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:04.630139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d505d-7764-4ada-9d37-272c6cad252d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:04.690984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:04.724209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:04.725388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:04.725671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:04.725945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:04.726670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:04.728422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:04.728422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:13:04.728422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:13:04.729752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:04.851908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 756/1497] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:13:04 2022] POST /identity/v3/projects => generated 377 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:13:04.856016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:04.857597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:04.892497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b0f42-81ff-40ea-84a6-1a239956fedd admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:04.986957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:04.987668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:04.987955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:04.988234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:04.988928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:04.990002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:04.990002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:13:04.990002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:13:04.991464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:05.091067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 742/1498] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:13:04 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:13:05.095210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:05.096726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:05.158981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e3415-f006-49b1-80c1-9ef10a6ba4a0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:05.244165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:05.254411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:05.255085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:13:05.260142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:05.272508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 757/1499] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:13:05.276555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:05.278398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:05.289914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b02340-7d7f-4e6f-8c7f-df553619bf27 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:05.315890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:05.385478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:05.386148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.386433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.386710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.387469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:05.388485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:05.388485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:13:05.388485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:13:05.389372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e620326-306f-4533-a90c-0847c403eda9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:05.402032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 743/1500] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:13:05.406193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:05.407752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:05.441749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c3835-98c1-487a-bee7-074ef5e9cbcc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:05.533888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:05.534697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.534984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.535295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] PATH_INFO: `/v3/projects/a7a1bbb3d13542f6bab497de4b97c03d/users/ff329c0ff2b24532ab188fe03dcc9810/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.557157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a1bbb3d13542f6bab497de4b97c03d, user_id=ff329c0ff2b24532ab188fe03dcc9810, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:05.558182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:05.558182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:13:05.558182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:13:05.560134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645321a-4c36-4f3c-883f-0232d0d5d4f6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:05.656529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 758/1501] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:13:05 2022] PUT /identity/v3/projects/a7a1bbb3d13542f6bab497de4b97c03d/users/ff329c0ff2b24532ab188fe03dcc9810/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:13:05.669455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f57aa-ec70-4c53-a0e9-affa86943d9a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.669851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f57aa-ec70-4c53-a0e9-affa86943d9a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.670133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f57aa-ec70-4c53-a0e9-affa86943d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.726512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2d5f57aa-ec70-4c53-a0e9-affa86943d9a None None] MFA Rules not processed for user `ff329c0ff2b24532ab188fe03dcc9810`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:05.727742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2d5f57aa-ec70-4c53-a0e9-affa86943d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:05.804198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 744/1502] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:13:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:13:05.817015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.817303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.817575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.819195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 759/1503] 10.222.0.59 () {54 vars in 963 bytes} [Thu Jul 14 11:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:13:05.823845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daaf12-7a6c-4e46-820b-1b83df9e54ad None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:05.824140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daaf12-7a6c-4e46-820b-1b83df9e54ad None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:05.824418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daaf12-7a6c-4e46-820b-1b83df9e54ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:05.854417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:05.900254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c3daaf12-7a6c-4e46-820b-1b83df9e54ad None None] MFA Rules not processed for user `c86f8e914f3f4312a60791af443b2f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:05.901455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c3daaf12-7a6c-4e46-820b-1b83df9e54ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:05.975590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 745/1504] 10.222.0.59 () {58 vars in 1060 bytes} [Thu Jul 14 11:13:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:13:05.983275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:05.984760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:06.034722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:06.088888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5bc48-d45d-4d87-ae24-d44d8cc340f2 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:06.161546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:06.162579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:06.162874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:06.163172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:06.164808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:06.264493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:06.265945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:06.269592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:06.284907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 760/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:06.327079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2adadb-ed13-4218-ad5d-0d806ba508bc None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:06.327518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2adadb-ed13-4218-ad5d-0d806ba508bc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:06.327667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2adadb-ed13-4218-ad5d-0d806ba508bc None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:06.328948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 746/1506] 10.222.0.59 () {52 vars in 829 bytes} [Thu Jul 14 11:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:13:06.332503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:06.333794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:06.433313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3eb1d-0b21-408d-81cb-3153de009062 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:06.524482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:06.525362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:06.525624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:06.525898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] PATH_INFO: `/v3/users/ff329c0ff2b24532ab188fe03dcc9810` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:06.537894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff329c0ff2b24532ab188fe03dcc9810)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:06.538913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:06.538913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:13:06.538913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:13:06.540154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:06.549584 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:06.759718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff329c0ff2b24532ab188fe03dcc9810'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:06.772688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff329c0ff2b24532ab188fe03dcc9810'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:06.792251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff329c0ff2b24532ab188fe03dcc9810'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:06.792736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff329c0ff2b24532ab188fe03dcc9810'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:06.866716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff329c0ff2b24532ab188fe03dcc9810'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:06.873343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 761/1507] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:13:06 2022] DELETE /identity/v3/users/ff329c0ff2b24532ab188fe03dcc9810 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:13:06.877831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:06.878150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:06.878458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:06.934512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:06.935761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:07.012290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:07.024928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 747/1508] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:13:07.034580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:07.035900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:07.137118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faf957-f518-4195-9393-53fa8c6298a6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:07.276987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:07.276987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:07.276987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:07.276987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:07.277495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:07.277495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:07.330057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:07.331569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ae00a-71cc-470b-9b17-bd97c6eca479 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:07.350995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 762/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:07.825235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:07.825235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:07.836668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d30058-07a9-4ef0-a015-182c983b8862 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:07.934830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:07.935733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:07.936022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:07.936298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] PATH_INFO: `/v3/projects/a7a1bbb3d13542f6bab497de4b97c03d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:07.947052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a1bbb3d13542f6bab497de4b97c03d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:07.948124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:13:07.948124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:13:07.948124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:13:07.949331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:08.082050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a1bbb3d13542f6bab497de4b97c03d', 'request_id': 'req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:08.088135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a1bbb3d13542f6bab497de4b97c03d', 'request_id': 'req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:08.101212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a1bbb3d13542f6bab497de4b97c03d', 'request_id': 'req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:08.204961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] The token cache is being invalidate because project a7a1bbb3d13542f6bab497de4b97c03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:13:08.205252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:08.205679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-fc93e958-4fcf-4d6f-bec4-17a263c3b91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:13:08.207172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 748/1510] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:13:07 2022] DELETE /identity/v3/projects/a7a1bbb3d13542f6bab497de4b97c03d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:13:17.375317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=62 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:18.202946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:51.611403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:51.612341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:51.612341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:51.614731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:14:51.621458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81f0d-9e84-4f4e-ad58-5d2c1d13986b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:51.622313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81f0d-9e84-4f4e-ad58-5d2c1d13986b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:51.622504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81f0d-9e84-4f4e-ad58-5d2c1d13986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:51.733088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-44f81f0d-9e84-4f4e-ad58-5d2c1d13986b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:51.734850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-44f81f0d-9e84-4f4e-ad58-5d2c1d13986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:51.871481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:14:52.035343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:52.037664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:52.189753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7152c8-06d4-4f2f-9b03-8ce001cad7b4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:52.196207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:52.196917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:52.197206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:52.197487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:52.207709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:52.209352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570a2e-42a2-45b1-8fae-1cb6fd6c52d1 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:52.221280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 764/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:14:53.131979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20b1fb-35d6-4735-8423-533004c662ec None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:53.139115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20b1fb-35d6-4735-8423-533004c662ec None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:53.139115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20b1fb-35d6-4735-8423-533004c662ec None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:53.139115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:14:53.139583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084e0-d9ba-4543-a489-a6fff5859b76 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:53.140224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084e0-d9ba-4543-a489-a6fff5859b76 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:53.140224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe084e0-d9ba-4543-a489-a6fff5859b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:53.161432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-cbe084e0-d9ba-4543-a489-a6fff5859b76 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:53.162682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-cbe084e0-d9ba-4543-a489-a6fff5859b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:53.172961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:53.349633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20b1fb-35d6-4735-8423-533004c662ec None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:53.368791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20b1fb-35d6-4735-8423-533004c662ec None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:53.377288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:53.378375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:53.378813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:53.379292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:53.396078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:53.398127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc373694-913c-4a60-8647-aa7c463cd056 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:53.409913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 751/1516] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:53.842018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50700b-f282-4b64-88a7-cb89ea806f4f None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:53.842432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50700b-f282-4b64-88a7-cb89ea806f4f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:53.842769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50700b-f282-4b64-88a7-cb89ea806f4f None None] PATH_INFO: `/v3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:53.844569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 766/1517] 10.222.0.67 () {54 vars in 925 bytes} [Thu Jul 14 11:14:53 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:14:53.852284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdfa27-003c-4c15-bbaa-925d0d62a352 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:53.852616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdfa27-003c-4c15-bbaa-925d0d62a352 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:53.852894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdfa27-003c-4c15-bbaa-925d0d62a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:53.930084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-13bdfa27-003c-4c15-bbaa-925d0d62a352 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:53.931372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-13bdfa27-003c-4c15-bbaa-925d0d62a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:53.955679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:54.020274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 752/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jul 14 11:14:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:03.155187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=56 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:03.489570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741938f2-caac-4f93-a455-af26845cff21 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:03.489860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741938f2-caac-4f93-a455-af26845cff21 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:03.490142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741938f2-caac-4f93-a455-af26845cff21 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:03.491536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 767/1519] 10.222.0.59 () {54 vars in 960 bytes} [Thu Jul 14 11:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:03.496390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681612-8358-4c8a-ad95-6351675dbca4 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:03.496854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681612-8358-4c8a-ad95-6351675dbca4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:03.496948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681612-8358-4c8a-ad95-6351675dbca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:03.515629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d1681612-8358-4c8a-ad95-6351675dbca4 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:03.516680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d1681612-8358-4c8a-ad95-6351675dbca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:03.525627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 753/1520] 10.222.0.59 () {58 vars in 1057 bytes} [Thu Jul 14 11:15:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:03.534091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741938f2-caac-4f93-a455-af26845cff21 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:03.547458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741938f2-caac-4f93-a455-af26845cff21 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:03.552413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:03.553078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:03.553365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:03.553651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:03.562683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:03.564335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1bcf2-b68a-443a-a47a-ce0a9b5e474f service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:03.574009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 768/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:04.724089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db412-c011-4111-a437-3d0c9923ae77 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:04.724624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db412-c011-4111-a437-3d0c9923ae77 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:04.724722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9db412-c011-4111-a437-3d0c9923ae77 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:04.726062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:04.730856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea02548-384f-4e5e-9d74-576e7484abad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:04.731371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea02548-384f-4e5e-9d74-576e7484abad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:04.731477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea02548-384f-4e5e-9d74-576e7484abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:04.750434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1ea02548-384f-4e5e-9d74-576e7484abad None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:04.751655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1ea02548-384f-4e5e-9d74-576e7484abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:04.759542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:04.915548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9db412-c011-4111-a437-3d0c9923ae77 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:04.930103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9db412-c011-4111-a437-3d0c9923ae77 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:04.935700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:04.936352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:04.936671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:04.936978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:04.945337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:04.948532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f586c6d-5ce7-4a47-91c7-c293f6c7b190 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:04.957965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 755/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:05.782194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882755d-b139-4d56-b4ab-b6ea64fa9eef None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:05.782739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882755d-b139-4d56-b4ab-b6ea64fa9eef None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:05.782854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882755d-b139-4d56-b4ab-b6ea64fa9eef None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:05.784192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:05.789017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a985-0dce-483c-8cc2-31f39ff18888 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:05.789442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a985-0dce-483c-8cc2-31f39ff18888 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:05.789588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78a985-0dce-483c-8cc2-31f39ff18888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:05.808069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4b78a985-0dce-483c-8cc2-31f39ff18888 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:05.809285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4b78a985-0dce-483c-8cc2-31f39ff18888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:05.817947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:05.970748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882755d-b139-4d56-b4ab-b6ea64fa9eef None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:05.986043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882755d-b139-4d56-b4ab-b6ea64fa9eef None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:05.991545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:05.992187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:05.992470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:05.992748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:06.001264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:06.002855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760d31-6a4e-435e-8c6c-2685862310a4 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:06.012226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 771/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:15:10.258221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b374-8334-4383-b13e-0099103924d7 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:10.258756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b374-8334-4383-b13e-0099103924d7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:10.258886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b374-8334-4383-b13e-0099103924d7 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:10.260257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:10.266252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e484-9433-4e17-beb2-43adf152c8d6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:10.266672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e484-9433-4e17-beb2-43adf152c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:10.267187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e484-9433-4e17-beb2-43adf152c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:10.285797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a663e484-9433-4e17-beb2-43adf152c8d6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:10.286991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a663e484-9433-4e17-beb2-43adf152c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:10.294914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:10.369272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b374-8334-4383-b13e-0099103924d7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:10.370951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2134b374-8334-4383-b13e-0099103924d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:10.454308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b374-8334-4383-b13e-0099103924d7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:10.457851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:10.458498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:10.458785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:10.459063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:10.468963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:10.470411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513193b-540e-4687-8eaf-8cccf0ce092e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:10.479856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 758/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:12.545816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f43ce6-a786-4bcb-b982-9f25d71ee040 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:12.546359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f43ce6-a786-4bcb-b982-9f25d71ee040 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:12.546359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f43ce6-a786-4bcb-b982-9f25d71ee040 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:12.547811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:12.553019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60165-8e1d-4acd-b1e9-cbd650aed030 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:12.553442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60165-8e1d-4acd-b1e9-cbd650aed030 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:12.553562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60165-8e1d-4acd-b1e9-cbd650aed030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:12.572098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-53a60165-8e1d-4acd-b1e9-cbd650aed030 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:12.573288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-53a60165-8e1d-4acd-b1e9-cbd650aed030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:12.581664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:12.618073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f43ce6-a786-4bcb-b982-9f25d71ee040 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:12.629202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f43ce6-a786-4bcb-b982-9f25d71ee040 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:12.632674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:12.633292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:12.633573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:12.633847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:12.642684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:12.644136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e287e5-d490-4b65-9bb3-f44c03134ee5 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:12.654511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 774/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:16.509068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:16.509648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:16.509757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:16.511048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:16.516850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3da8-46b0-4c53-a294-2f744daf81c5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:16.517315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3da8-46b0-4c53-a294-2f744daf81c5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:16.517418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3da8-46b0-4c53-a294-2f744daf81c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:16.535275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-684a3da8-46b0-4c53-a294-2f744daf81c5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:16.536479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-684a3da8-46b0-4c53-a294-2f744daf81c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:16.544605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:16.847375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:16.849258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:16.915598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:16.929734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b5d68-41dd-4332-bfed-be4dac685870 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:16.933308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:16.933950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:16.934233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:16.935506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:16.944967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:16.946387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d86fc1-2310-4412-9d6c-b01cd096b065 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:16.956713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 761/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:19.022236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.022778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.022778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.024230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:15:19.029525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016aa1d-9b0c-4fe2-a997-3f1d788052ee None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.029959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016aa1d-9b0c-4fe2-a997-3f1d788052ee None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.030069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016aa1d-9b0c-4fe2-a997-3f1d788052ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.048848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d016aa1d-9b0c-4fe2-a997-3f1d788052ee None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:19.050044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d016aa1d-9b0c-4fe2-a997-3f1d788052ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.057898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:15:19.374593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.376279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.390332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee864f-31af-41ab-bb3a-c9db914d8e3b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.394184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.394818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.395102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.395407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.404549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.406315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac0a8f-fd55-40c7-86f5-be6ddb593781 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.416315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 777/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:29.043538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=11 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:29.412072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=33 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:53.824184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36993b2d-04ae-4fa7-864c-58ad8a51deab None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.824896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36993b2d-04ae-4fa7-864c-58ad8a51deab None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.824896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36993b2d-04ae-4fa7-864c-58ad8a51deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.850916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-36993b2d-04ae-4fa7-864c-58ad8a51deab None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:53.852438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-36993b2d-04ae-4fa7-864c-58ad8a51deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:53.854521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de27-d41a-43c8-8765-fcd82c4739b9 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.855588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de27-d41a-43c8-8765-fcd82c4739b9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.856105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de27-d41a-43c8-8765-fcd82c4739b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.863940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 763/1540] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:53.881257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.881699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.882821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.890711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-80a8de27-d41a-43c8-8765-fcd82c4739b9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:53.892010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-80a8de27-d41a-43c8-8765-fcd82c4739b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:53.907268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 778/1541] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:16:53.910416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.910744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.910995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.916456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:53.917684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:53.932400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:53.932634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 764/1542] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:53.933803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:53.936994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:53.946834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 779/1543] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:16:53.952252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:53.954591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf6df-76f2-423b-8e41-f49120fa9065 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:53.958557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678e1f4-fd17-409d-a357-443a79a3a746 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:53.964894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:53.965601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.965894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.966178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.966933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:53.967300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:53.968564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:16:53.968564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:16:53.968564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:16:53.969499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:53.973957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:53.977887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:53.977887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:53.977887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:53.977887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:53.977887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:53.978798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:53.987265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 780/1544] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:53.993513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:53.998907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 765/1545] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.007302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.011147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fd8e2-60ba-489e-93c6-667a2de41fda admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.024694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.025380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.025670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.025950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.026655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.031028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa856-1f92-4cc3-bb0d-c607fd46095c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.032384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016fb9a-ffad-4daa-9291-97d83038cbf7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.041926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.042608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.042890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.043189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.043888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.044924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:16:54.044924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:16:54.044924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:16:54.046126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a18c-cf53-4121-bc74-a7a9e75dbb29 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.290666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 781/1546] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:53 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:16:54.292247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.292550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.292835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.322678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.324355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.346555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 766/1547] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/projects => generated 371 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:54.347941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a812-626b-416e-a586-6801b944deeb None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.348234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a812-626b-416e-a586-6801b944deeb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.348508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02a812-626b-416e-a586-6801b944deeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.382609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fc02a812-626b-416e-a586-6801b944deeb None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.383878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fc02a812-626b-416e-a586-6801b944deeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.411643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 782/1548] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.411643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.424370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d63519-e5e3-4a40-b5fe-7e80ba37c336 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.458851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 767/1549] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.460738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93a2c-7573-4016-81b9-994aa5ddef81 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.460738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93a2c-7573-4016-81b9-994aa5ddef81 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.460959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f93a2c-7573-4016-81b9-994aa5ddef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.479362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d3f93a2c-7573-4016-81b9-994aa5ddef81 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.480868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d3f93a2c-7573-4016-81b9-994aa5ddef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.488336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.489222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 768/1550] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.489363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.489363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.489651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.490297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.490405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.490598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.490881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.492049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:16:54.492049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:16:54.492049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:16:54.493650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eabf3c-0031-4c63-8f32-6216d9085e5a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.514106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.520693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.524542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 769/1551] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.525356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.536520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b1c7-4f03-448f-b323-1aaee70ad15c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.542099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.542772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.543056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.543363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.544056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.545984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c5efb-045b-4a14-ae0a-5e790b0fccdb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.587264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 783/1552] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:54.588556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79921d-19d7-494b-af5d-deecdfc815f3 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.588849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79921d-19d7-494b-af5d-deecdfc815f3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.589133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79921d-19d7-494b-af5d-deecdfc815f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.606802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9a79921d-19d7-494b-af5d-deecdfc815f3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.608088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9a79921d-19d7-494b-af5d-deecdfc815f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.617109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 784/1553] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.618371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4823-b989-4c03-8856-5e2562545e21 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.618660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4823-b989-4c03-8856-5e2562545e21 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.618939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4823-b989-4c03-8856-5e2562545e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.637555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d5bb4823-b989-4c03-8856-5e2562545e21 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.638734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5bb4823-b989-4c03-8856-5e2562545e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.646605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 785/1554] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.647885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cb89-0cac-4345-9984-97182820dc77 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.648318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cb89-0cac-4345-9984-97182820dc77 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.648603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cb89-0cac-4345-9984-97182820dc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.655764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 770/1555] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:54.656995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889430fd-aaf6-43b2-a045-35bbdfdd0974 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.658018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889430fd-aaf6-43b2-a045-35bbdfdd0974 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.658018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889430fd-aaf6-43b2-a045-35bbdfdd0974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.668171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ade0cb89-0cac-4345-9984-97182820dc77 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.669360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ade0cb89-0cac-4345-9984-97182820dc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.677350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-889430fd-aaf6-43b2-a045-35bbdfdd0974 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.677730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 786/1556] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.679104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-889430fd-aaf6-43b2-a045-35bbdfdd0974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.679634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.679918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.680194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.690918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 771/1557] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.692423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94493-05c6-4422-8340-175788335572 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.692720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94493-05c6-4422-8340-175788335572 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.693001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94493-05c6-4422-8340-175788335572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.699907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.701118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.709365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 787/1558] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.710496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.712983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-94b94493-05c6-4422-8340-175788335572 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.714472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-94b94493-05c6-4422-8340-175788335572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.720173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a22207-f2de-43ab-a64e-4247cdd2e9a7 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.723376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 772/1559] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.724933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.725299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.725478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.725602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.727779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.727779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.727779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.727779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.729360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.731182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:54.742531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 788/1560] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:54.743686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.745059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.746654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.754562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532ad7b-a71d-495a-a401-95a76a951cd7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.755028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 773/1561] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.755929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.763493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.764347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.764725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.765050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.765814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.766127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.767556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926a4f-ceba-460d-821a-45d3c4b25ff6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.767810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d30774-8665-4cf8-9f75-6d6a18fa2bf0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.772323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.773211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.773582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.773863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.774627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.776302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.780666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 789/1562] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.781997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.782292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.782571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.789920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 774/1563] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:54.791062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.801367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:54.807040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623c52e-8a43-4fb0-b252-48c7d912469f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.807190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:54.807287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.807389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.807662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.807942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.808663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.808995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.810603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.811289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 790/1564] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:54.812079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.826658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff28fb2-5823-4da8-abc1-ff98b6d038d4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.826918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 775/1565] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.827750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.834329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.835079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.835398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.835755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.836502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.836815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.838438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.840488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c54c5-884c-4554-a441-1af756d4c347 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.845256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.846156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.846275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.846553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.847421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.847775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.849615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.856312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 791/1566] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.856312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.867618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea200eb-8c25-4e02-9f5f-4bc6a1454570 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.872530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 776/1567] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.873355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.873522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.874042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.874427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.874718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.875496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.877117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.884817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b091-76be-4a1a-90c0-7dc7b115a71f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.889783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.890921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.890921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.891204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.891978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.892246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.893869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.895049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 792/1568] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:54.898838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.909095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9fad-4e04-4555-9213-4f91df803d8f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.909894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 777/1569] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.911193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.915021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.915749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.916034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.916314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.917014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.918944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.922962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:54.930322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53be8fd-4803-4f44-a3e9-3fd145c1a3c8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.936198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.936883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.937163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.937447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.938222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.939896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.950777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 778/1570] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:54.955203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.962010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8daa-68e1-4e5f-87f1-386feed5aab7 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:54.966788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:54.967483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:54.967805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:54.968083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:54.968796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:16:54.969104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:54.970690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:54.984806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 779/1571] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:16:54.985688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:54.996677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a798242-652a-4e70-8d10-82c4ec456af6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.001175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.001799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.002189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.002785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.003504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.007705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.014408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 793/1572] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.014408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.081158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ce36-17bf-4253-8d55-438b0c4bee9c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.086468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.087146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.087440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.087713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.088389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.090286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.142841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 780/1573] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:54 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.143689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.203673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 794/1574] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.203673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.209060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241facf-af82-4d69-bad8-253da59c0664 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.263592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.264259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.264540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.264814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.265316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354fdc4-1bb2-4030-9b89-b59be384f999 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.265503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.267438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.270082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.270960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.271277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.271552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] PATH_INFO: `/v3/projects/457a247df22a45b39318dd2add051120/users/2c8ce3b0a82f47dc9ae6a4fb2abcaeaa/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.286898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=457a247df22a45b39318dd2add051120, user_id=2c8ce3b0a82f47dc9ae6a4fb2abcaeaa, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.289810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.354071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 781/1575] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/projects => generated 393 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.354885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.382425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:55.388499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 795/1576] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:55 2022] PUT /identity/v3/projects/457a247df22a45b39318dd2add051120/users/2c8ce3b0a82f47dc9ae6a4fb2abcaeaa/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:55.392143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.411288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:55.424393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760d5c-3b13-48ae-8b72-770b69bc0fd5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.451565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958c26-bdc4-4db5-91e5-3745b8257a3f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.456322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.457211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.457496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.457774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] PATH_INFO: `/v3/projects/4be885d3b724429fb72f06cd7750ecf1/users/a97fc05129a940b0a92832a1e9ebf4ee/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.473507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be885d3b724429fb72f06cd7750ecf1, user_id=a97fc05129a940b0a92832a1e9ebf4ee, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.476630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.481923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.482600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.482889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.483202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.483895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.485834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.575692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 796/1577] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:55 2022] PUT /identity/v3/projects/4be885d3b724429fb72f06cd7750ecf1/users/a97fc05129a940b0a92832a1e9ebf4ee/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:55.576586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.603791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 782/1578] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.603791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.642029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9806f8-ee73-4375-a8f5-5886f55f9a56 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.661435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f1aae-edd7-4683-84cb-f41329e072f4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.666756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.667471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.667764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.668047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.668742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.671003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.700056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.700803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.701086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.701377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.702036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.703960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.815939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 783/1579] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.816875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.828602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3868d-4f10-4bd9-be1c-b41bf521986d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.833349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.834032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.834319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.835293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.836066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.838122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.845824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 797/1580] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/projects => generated 367 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.847309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.864004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:55.918578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e01d-12c7-4b5d-b030-cdfd6de94c1f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.922959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.923666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.923948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.924228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.924943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.926887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fddd4c-6c6d-495c-8597-7b0313abbd20 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:55.934197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 784/1581] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:55.934197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:55.945157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b8c70-57fc-452f-b440-a69a2ebf8800 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:55.949829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:55.950524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:55.950803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:55.951085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:55.951823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:55.953749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea815a85-4ddc-4dc1-817a-36d01324d809 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.033147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 798/1582] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.036365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 785/1583] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:55 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.036549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.036549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.036549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.036894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.036894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.036894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.055333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:56.056591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:56.064581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 799/1584] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:56.065386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.065896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] MFA Rules not processed for user `a97fc05129a940b0a92832a1e9ebf4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:56.067104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:56.077166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664a69f-2ae4-4989-9246-3195f4191fc5 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.082053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.082807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.083098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.083400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.084093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.086517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.095946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:56.143642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 786/1585] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:56.144543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.155816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed408874-d432-4832-b187-eeff9a45e537 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.161233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.161233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.166830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.166830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.166830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.166830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.174758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 787/1586] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.175630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.186872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4082-9f72-4f01-9f46-21f2c2135e08 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.191666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.192342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.192628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.192911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.193600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.196226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.199954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 800/1587] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.201349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.213964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55df7-aa02-4a3b-8036-3311b1156611 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.218645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.219387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.219672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.219943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.220660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.222246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.232676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 801/1588] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.233455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.244699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6ae8-5227-43ba-b86d-d0f8d8032835 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.249451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.250087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.250371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.250649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.251401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.252974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.263867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 802/1589] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.264660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.274117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97932ff-3ce7-4dce-a4b8-ce7b31e631be admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.278690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.279357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.279644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.279925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.280638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.282245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.292386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 803/1590] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.293153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.296506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:56.342158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 788/1591] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.342904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.354846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a23872-73f4-4356-893c-af74a8d65a39 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.361314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.361552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f3089-676b-425f-ba15-1e7895f1fadb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.361991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.362386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.362585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.363418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.365363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.367320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.368393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.368982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.369266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.380749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.382928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.393107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 804/1592] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:56.393937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.404301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274b1e-8165-4fd5-a039-15ddd4804e71 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.409630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.410341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.410594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.410869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.411625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.414615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.425303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 805/1593] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.429952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.437566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6230a-21fe-4a62-b3ef-6e7c88ea4a21 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.443626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.444945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.445231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.445508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.446242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.447959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.458369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 806/1594] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.459244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.474905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437894b8-6413-48ed-8bb5-e5bad89e65d9 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.483488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.489196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.489196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.489196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.489196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.489546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 789/1595] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.489546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.489728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.500788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 807/1596] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.501743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.512252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc3558-38ba-4395-885e-267645977625 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.533366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:56.572830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc93bee-996e-4e38-be61-a89d5827dd84 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.578694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.579481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.579807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.579981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.580742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.582682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.586854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.587677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.588028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.588366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.589170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.591026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.596206 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 790/1597] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.598825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.603325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 808/1598] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.604097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.611043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2704c1-68fe-4c95-ab0f-9ced91dcba07 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.617198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8515a-e169-40a5-bc98-c008b7984171 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.617620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.618337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.618624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.618994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.619687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.621317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.624712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.624712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.624712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.624712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] PATH_INFO: `/v3/projects/d71ac221cd034943aad67230e8b108cb/users/f36a80d102cb463889bde77cb83acf15/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.632531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 791/1599] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.633181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.640374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71ac221cd034943aad67230e8b108cb, user_id=f36a80d102cb463889bde77cb83acf15, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.643535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.644247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b791a-af9b-43da-baea-f5a93d99b8e4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.649749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.659682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.659682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.659682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.659682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.659682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.664812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 792/1600] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:56.665725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.675544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8fb00-6e0d-4feb-a82c-36753807e087 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.681065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.681785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.682105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.682385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.683092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.685157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.794396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 809/1601] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:56 2022] PUT /identity/v3/projects/d71ac221cd034943aad67230e8b108cb/users/f36a80d102cb463889bde77cb83acf15/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:56.795487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.873124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaf753-5aae-4e60-8b97-0e24a2adf854 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.877696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.883032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 793/1602] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:56 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:56.883509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.883509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.883509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] PATH_INFO: `/v3/projects/debc507273144e218f7c398e615bef01/users/3e22a5abdc2547d5acb7c5034f6983a9/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.883763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:56.892565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b749b7-87ed-492d-b750-ca9370c023f2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:56.896795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=debc507273144e218f7c398e615bef01, user_id=3e22a5abdc2547d5acb7c5034f6983a9, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.898106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:56.899019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:56.899333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:56.899617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] PATH_INFO: `/v3/projects/08cc50e4caea44e2a2bb1413a35e3b6d/users/e3e8aa5fc932453e95080fa836dfac0c/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:56.899749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:56.915497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cc50e4caea44e2a2bb1413a35e3b6d, user_id=e3e8aa5fc932453e95080fa836dfac0c, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:56.916537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:16:56.916537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:16:56.916537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:16:56.918510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.027670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 810/1603] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:56 2022] PUT /identity/v3/projects/debc507273144e218f7c398e615bef01/users/3e22a5abdc2547d5acb7c5034f6983a9/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.028350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 794/1604] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:56 2022] PUT /identity/v3/projects/08cc50e4caea44e2a2bb1413a35e3b6d/users/e3e8aa5fc932453e95080fa836dfac0c/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.029006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.037934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.052668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:57.117253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1b865-557d-4660-9ea2-67d3e2173ab9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.118458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a53b-e6a9-4df4-8cad-2eb7dc6019e0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.121990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.122654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.122945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.123310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.123788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.124014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.124683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.124970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.125248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] PATH_INFO: `/v3/projects/0ac8b98f65494486a1b109c0bc4b9b52/users/2c12aa6fbe0f4ff2af596fff374e423b/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.125747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.141534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 795/1605] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:57.141534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.150150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac8b98f65494486a1b109c0bc4b9b52, user_id=2c12aa6fbe0f4ff2af596fff374e423b, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.152072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a7d97-3c5e-40f0-b0a9-6c64c4372c20 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.153097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179877df-b908-4b35-80d0-4fcb1fdb3f0b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.155682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:57.157410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.158339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.158628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.158910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] PATH_INFO: `/v3/projects/f33c57605a974b44a62eebbdaba23ade/users/3ac486b2d0ac42f9ba2fed7b17418b7a/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.176106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33c57605a974b44a62eebbdaba23ade, user_id=3ac486b2d0ac42f9ba2fed7b17418b7a, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.178896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.300779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 811/1606] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:57 2022] PUT /identity/v3/projects/0ac8b98f65494486a1b109c0bc4b9b52/users/2c12aa6fbe0f4ff2af596fff374e423b/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.302078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91fcb-a13e-4c93-b08f-240513100133 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.302365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91fcb-a13e-4c93-b08f-240513100133 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.302641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91fcb-a13e-4c93-b08f-240513100133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.328639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c1a91fcb-a13e-4c93-b08f-240513100133 None None] MFA Rules not processed for user `f36a80d102cb463889bde77cb83acf15`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.329845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c1a91fcb-a13e-4c93-b08f-240513100133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.343646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 796/1607] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:57 2022] PUT /identity/v3/projects/f33c57605a974b44a62eebbdaba23ade/users/3ac486b2d0ac42f9ba2fed7b17418b7a/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.344913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.400099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 812/1608] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.404781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.405079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.405359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.414561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c17a5-828c-4148-a739-5f6ec79c6765 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.419633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.420355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.420782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.421076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.421816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.423668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdd3d9-3c34-456a-962b-8ac7b88dce62 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.423949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.425151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.435384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 797/1609] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:57.436597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0ab3d-3f18-4c4f-a7b2-94ed65252eac None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.436888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0ab3d-3f18-4c4f-a7b2-94ed65252eac None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.437159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0ab3d-3f18-4c4f-a7b2-94ed65252eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.438652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 813/1610] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.439841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.456979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64734c-96a7-405d-9e2c-3379a1fa7dab None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.456979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.456979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.456979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.457364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] PATH_INFO: `/v3/projects/cc04e272fd2e4559bc35a25ed1252c23/users/3d63b01c29064fb98655a558b5cd527f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.470117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dda0ab3d-3f18-4c4f-a7b2-94ed65252eac None None] MFA Rules not processed for user `3e22a5abdc2547d5acb7c5034f6983a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.471717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dda0ab3d-3f18-4c4f-a7b2-94ed65252eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.483625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc04e272fd2e4559bc35a25ed1252c23, user_id=3d63b01c29064fb98655a558b5cd527f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.486336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684763b-e13f-4486-b1cf-cc824c3fd300 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.544910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 798/1611] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.546356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.546651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.546932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.566528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 814/1612] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:57 2022] PUT /identity/v3/projects/cc04e272fd2e4559bc35a25ed1252c23/users/3d63b01c29064fb98655a558b5cd527f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.575290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.575290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.575290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.585786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] MFA Rules not processed for user `2c12aa6fbe0f4ff2af596fff374e423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.587075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.593080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.594258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.655240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 799/1613] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.656002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.680672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 815/1614] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.680672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.689079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4bd31-37d0-45a9-9fd2-80381f35a710 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.696416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.697499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.697911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.698202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] PATH_INFO: `/v3/projects/1d6afac67e114813807a04a29e64dfa6/users/02b231b457af4b83bd7169e7afd88889/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.718969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6afac67e114813807a04a29e64dfa6, user_id=02b231b457af4b83bd7169e7afd88889, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.721844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:57.722836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.741129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0843d8-ba1f-4cb2-8036-00bf1818db75 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.745958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.750329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.750329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.750329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.750329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:57.750329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279ace4-5338-466c-85d3-96eece732f5b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:57.815343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 816/1615] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:57 2022] PUT /identity/v3/projects/1d6afac67e114813807a04a29e64dfa6/users/02b231b457af4b83bd7169e7afd88889/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:57.816182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.846813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 800/1616] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:57.846813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dd9-063d-4a2e-886f-5fd4410c13fd None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.846813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dd9-063d-4a2e-886f-5fd4410c13fd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.846813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f671dd9-063d-4a2e-886f-5fd4410c13fd None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.847661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 801/1617] 10.222.0.59 () {54 vars in 961 bytes} [Thu Jul 14 11:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:57.848912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73854c-d089-4e91-b203-7a7d7a9cbc6f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.849453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73854c-d089-4e91-b203-7a7d7a9cbc6f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.849728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73854c-d089-4e91-b203-7a7d7a9cbc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.875497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2f932-ead4-427f-af3f-e595e83efe5e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.876096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0c73854c-d089-4e91-b203-7a7d7a9cbc6f None None] MFA Rules not processed for user `3d63b01c29064fb98655a558b5cd527f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:57.877073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0c73854c-d089-4e91-b203-7a7d7a9cbc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:57.925181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.925853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.926191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.926443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.942474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 802/1618] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:57.944674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e45ed-39a9-446b-8627-b068422ca04a None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.944971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e45ed-39a9-446b-8627-b068422ca04a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.945249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e45ed-39a9-446b-8627-b068422ca04a None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:57.946564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 803/1619] 10.222.0.59 () {54 vars in 961 bytes} [Thu Jul 14 11:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:57.947608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e45ed-39a9-446b-8627-b068422ca04a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:57.958676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e45ed-39a9-446b-8627-b068422ca04a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:57.963350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:57.964187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:57.965809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:57.966112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.005272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.007362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.018819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 817/1620] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:58.020156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.028272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:58.032162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98846c16-799f-480e-b6ea-30de59df68eb service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.037256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.038348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.038617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.038874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.039650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.042286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.046402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.048085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190006e-f8da-4fd1-bd45-477e2d5ea07d service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.057073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 818/1621] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:58.057990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.062430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 804/1622] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:58.067747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb7d9-f492-4a5c-b7ef-71f180357e94 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.070437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb7d9-f492-4a5c-b7ef-71f180357e94 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.070437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb7d9-f492-4a5c-b7ef-71f180357e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.076117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9626cb-9f84-49bf-afcb-01ef23ec0144 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.082214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.083094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.083593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.084080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.085650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.088149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.118098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ee4fb7d9-f492-4a5c-b7ef-71f180357e94 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:58.119658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ee4fb7d9-f492-4a5c-b7ef-71f180357e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:58.194821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 805/1623] 10.222.0.59 () {58 vars in 1058 bytes} [Thu Jul 14 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:58.196619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.196911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.197217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.215917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:58.217147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:58.225286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 819/1624] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:16:58 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:16:58.226926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.241791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 806/1625] 10.222.0.59 () {58 vars in 1058 bytes} [Thu Jul 14 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:58.247381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.270349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3be44-cd3d-4926-8b8b-a81058064d9e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.288889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.290122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.290416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.290703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] PATH_INFO: `/v3/projects/1d6afac67e114813807a04a29e64dfa6/users/02b231b457af4b83bd7169e7afd88889/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.294180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6afac67e114813807a04a29e64dfa6, user_id=02b231b457af4b83bd7169e7afd88889, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.299662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.305612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c046-af9a-4b1e-8248-c789c2f319af admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.322992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.323851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.324412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.324527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.329867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 807/1626] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:58 2022] PUT /identity/v3/projects/1d6afac67e114813807a04a29e64dfa6/users/02b231b457af4b83bd7169e7afd88889/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:58.330962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.335792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.337573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.349973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 820/1627] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:58.351206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.365307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370ee9-01c7-460b-ace1-de6db1058f0b service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.440488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b439f-e526-4244-9bca-e1bb643d8eab admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.446739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.447514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.447800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.448082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.448842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.450444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.451309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.452719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.453314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.453700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.463659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 821/1628] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:58.467603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.482133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e989-108c-4fea-b80f-e2c43db02427 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.487917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.488572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.488863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.489146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.526473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.528133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689585ff-25d4-404b-90c4-05a193bf7a0d service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.538715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 808/1629] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:58.543100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.543100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.543100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.552231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.555723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.567569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 822/1630] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:58.569708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.573778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] MFA Rules not processed for user `02b231b457af4b83bd7169e7afd88889`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:58.577147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:58.580833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22172b1-070f-4703-a8c4-e71b0e59c278 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.586146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.587050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.587524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.587818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] PATH_INFO: `/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c/users/ea9c05017b45471daa114c149cc848e3/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.624287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3c67aaebda4a3ea5e12a88d0d1063c, user_id=ea9c05017b45471daa114c149cc848e3, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.628228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.733556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 823/1631] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:58 2022] PUT /identity/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c/users/ea9c05017b45471daa114c149cc848e3/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:58.737184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.775110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 809/1632] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:58.788939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.906657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676b82-c684-40ef-bcda-11a64c6a3127 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.914946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.915299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.915458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.915790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.916769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:58.918761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:58.932815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 824/1633] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:16:58.934874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:58.945952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:58.970139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc9b9e-b630-4c7a-9cca-291fc93f7fa1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.972566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340441c2-8d3c-4d7d-a555-947f45eaf897 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:58.986304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:58.988122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:58.989284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:58.989976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] PATH_INFO: `/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c/users/ea9c05017b45471daa114c149cc848e3/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:58.996787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3c67aaebda4a3ea5e12a88d0d1063c, user_id=ea9c05017b45471daa114c149cc848e3, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:59.000763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d6d09-5e6c-4208-8a78-c16d82e482ba admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:59.052843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 825/1634] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:16:58 2022] PUT /identity/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c/users/ea9c05017b45471daa114c149cc848e3/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:16:59.060220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead83b3d-cb69-4e8a-a315-e9f9f158e411 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.061446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead83b3d-cb69-4e8a-a315-e9f9f158e411 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.061732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead83b3d-cb69-4e8a-a315-e9f9f158e411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.141998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:59.150851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ead83b3d-cb69-4e8a-a315-e9f9f158e411 None None] MFA Rules not processed for user `ea9c05017b45471daa114c149cc848e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:59.152092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ead83b3d-cb69-4e8a-a315-e9f9f158e411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:59.156540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:59.157213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.157652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.157940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.305465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 826/1635] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:59.308728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d80-a57f-46ee-8b38-63b2d17231ad None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.309458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d80-a57f-46ee-8b38-63b2d17231ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.309927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d80-a57f-46ee-8b38-63b2d17231ad None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.312845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 827/1636] 10.222.0.59 () {54 vars in 912 bytes} [Thu Jul 14 11:16:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:59.314089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:59.314804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26398266-8528-4826-aaab-c9b831ce9a9c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.315314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26398266-8528-4826-aaab-c9b831ce9a9c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.315768 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26398266-8528-4826-aaab-c9b831ce9a9c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.318357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 828/1637] 10.222.0.59 () {54 vars in 912 bytes} [Thu Jul 14 11:16:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:59.319217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2c73-2939-42fb-a577-a1a8b15de7e1 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:59.321171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.321631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.322084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.346363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 810/1638] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:59.348079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:59.350830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:59.352769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.353073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.353475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.376789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:59.378675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:59.487209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 811/1639] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:59.526564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 829/1640] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:59.561537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:59.648166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228a34e-a505-4912-8d9c-77093ec7de19 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:59.654175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:59.661658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:59.663022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.663560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.663992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.678856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:59.681503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:59.698847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 812/1641] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:59.719439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:59.849556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:59.861135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251a088-fe93-42c2-9fc0-e75ae572b93f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:59.866347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:59.867564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.867705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.868024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.879384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:59.880914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:59.895536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 830/1642] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:59.972647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:00.054646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f398c-2b8d-4436-80ac-b60ab645262d service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:00.054646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:00.054646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:00.054646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:00.090076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:00.090076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:00.090076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:00.090076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 813/1643] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:01.222901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:01.236232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528d32b-247a-46a2-942b-cedbdb26aed5 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:01.242822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:01.243814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:01.244323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:01.244711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:01.318161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:01.320335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7f376-9bda-424f-90cc-6b9d3d5e6b98 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:01.333592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 831/1644] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:01.679589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:01.694445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:01.843868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597f829-4a53-4461-93ca-885b91f2981d service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:01.857881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:01.859100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:01.859560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:01.860002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] PATH_INFO: `/v3/users/3e22a5abdc2547d5acb7c5034f6983a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:01.861981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e22a5abdc2547d5acb7c5034f6983a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:01.865592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: WARNING py.warnings [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:17:01.865592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: warnings.warn(msg) Jul 14 11:17:01.865592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: Jul 14 11:17:01.870914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:02.184842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e22a5abdc2547d5acb7c5034f6983a9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:02.185566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e22a5abdc2547d5acb7c5034f6983a9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:02.248641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e22a5abdc2547d5acb7c5034f6983a9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:02.272798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e22a5abdc2547d5acb7c5034f6983a9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:02.369166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e22a5abdc2547d5acb7c5034f6983a9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:02.376636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 814/1645] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:01 2022] DELETE /identity/v3/users/3e22a5abdc2547d5acb7c5034f6983a9 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:02.381849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:02.382809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:02.383328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:02.396077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:02.429936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:02.434721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:02.442304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:02.580794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 832/1646] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:02.598222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:02.600807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:02.738584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c7463-19f5-4ac6-aa77-90d484593690 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:02.744386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:02.745229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:02.745520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:02.746002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:02.757724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:02.759392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d092406-233e-49bd-82af-a7632ffb3273 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:02.771995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 815/1647] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:03.340295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:03.342957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:03.379201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2450e-4be0-43ca-9652-d1490088b913 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:03.402324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:03.404371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:03.405059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:03.405573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] PATH_INFO: `/v3/users/a97fc05129a940b0a92832a1e9ebf4ee` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:03.407227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97fc05129a940b0a92832a1e9ebf4ee)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:03.417539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:03.743586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97fc05129a940b0a92832a1e9ebf4ee'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:03.769345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97fc05129a940b0a92832a1e9ebf4ee'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:03.810211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97fc05129a940b0a92832a1e9ebf4ee'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:03.813097 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97fc05129a940b0a92832a1e9ebf4ee'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:03.901397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97fc05129a940b0a92832a1e9ebf4ee'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:03.909083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 833/1648] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:03 2022] DELETE /identity/v3/users/a97fc05129a940b0a92832a1e9ebf4ee => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:03.914769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:03.924006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:03.924006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:03.956127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:03.956127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:04.004621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:04.057785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 816/1649] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:04.103940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:04.106591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:04.120028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:04.121676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:04.145624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff43855-2cbb-48b9-a4b4-7d29fb8256e1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:04.172707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:04.196394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.196692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.197788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] PATH_INFO: `/v3/projects/debc507273144e218f7c398e615bef01` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.201931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=debc507273144e218f7c398e615bef01)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:04.214067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:04.275545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed807a-21a8-40ce-81ac-a7ae6b06eafb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:04.284505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:04.285197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.285487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.287859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.315322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:04.317434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dacda-f1b5-4ebe-911a-16b8cff47e19 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:04.351615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 834/1650] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:04.390235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'debc507273144e218f7c398e615bef01', 'request_id': 'req-524f45da-525e-44c6-9d1a-03601a28ff1b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:04.414217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'debc507273144e218f7c398e615bef01', 'request_id': 'req-524f45da-525e-44c6-9d1a-03601a28ff1b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:04.435461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'debc507273144e218f7c398e615bef01', 'request_id': 'req-524f45da-525e-44c6-9d1a-03601a28ff1b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:04.553390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] The token cache is being invalidate because project debc507273144e218f7c398e615bef01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:04.553784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:04.555527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-524f45da-525e-44c6-9d1a-03601a28ff1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:04.562177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 817/1651] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:04 2022] DELETE /identity/v3/projects/debc507273144e218f7c398e615bef01 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:04.564037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.564512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.564713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.605822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:04.609103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:04.665835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:04.760312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 835/1652] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:04.768092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.768730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.769239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.801242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:04.807772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:04.826354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 818/1653] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:04.830580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:04.857461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca853fd-1d38-4937-8184-48251fee7c69 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:04.870199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:04.871270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.872809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.873219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.876635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:04.877075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:04.881913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:04.909755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 836/1654] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:04.911236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:04.934592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f40c37-91b5-454b-91ab-272af2711f7b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:04.949126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:04.951155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:04.951862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:04.952421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:04.953576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:04.957790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:05.074275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 819/1655] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:04 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:05.079826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:05.105732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d3a99-fc3f-4c0f-8184-66dab43aff4d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:05.259546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:05.260305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:05.260781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:05.261063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:05.262288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:05.265272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:05.424826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 837/1656] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:05 2022] POST /identity/v3/users => generated 290 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:05.429315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:05.444464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba395fdf-ee33-4c6f-804f-d9e7ad79daba admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:05.452412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:05.453549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:05.454040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:05.454538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:05.455489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:05.457760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:05.484137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 820/1657] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:05.489657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:05.505027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22ec66-fa86-4d6f-b01e-5fa6421a4b76 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:05.517176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:05.519558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:05.519859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:05.520146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:05.520949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:05.523369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d576aa-cc3d-488f-8e94-e213db93f7a2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:05.558681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 838/1658] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:05.561662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:05.578011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a9c9-6c1a-4803-9bb0-c71549d18acc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:05.607261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:05.608303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:05.608663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:05.609015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] PATH_INFO: `/v3/projects/6f2f4c8ea800491e9b6b4aff2dd75f4e/users/788cf14f3ce440ec8ff3669e29a5fa4a/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:05.653960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2f4c8ea800491e9b6b4aff2dd75f4e, user_id=788cf14f3ce440ec8ff3669e29a5fa4a, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:05.662234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:05.787917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 821/1659] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:05 2022] PUT /identity/v3/projects/6f2f4c8ea800491e9b6b4aff2dd75f4e/users/788cf14f3ce440ec8ff3669e29a5fa4a/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:05.801811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a478c0-b861-4144-9037-78867e564b2b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:05.802412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a478c0-b861-4144-9037-78867e564b2b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:05.802507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a478c0-b861-4144-9037-78867e564b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:05.868833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d9a478c0-b861-4144-9037-78867e564b2b None None] MFA Rules not processed for user `788cf14f3ce440ec8ff3669e29a5fa4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:05.869375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d9a478c0-b861-4144-9037-78867e564b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.014808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 839/1660] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:06.016268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:06.111325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:06.111846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:06.115909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:06.212417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f7968-6e4a-41e9-8728-3589d3d8b902 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:06.219464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] MFA Rules not processed for user `2c8ce3b0a82f47dc9ae6a4fb2abcaeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:06.219946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:06.220906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:06.221472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:06.221944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:06.222915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:06.224589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.226439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:06.370000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 822/1661] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:06 2022] POST /identity/v3/projects => generated 379 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:06.371015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:06.374111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.407990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 840/1662] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:06.408963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:06.449086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768741b-6df1-4470-afd8-a1f6cbb9e3ad None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:06.526167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52f10-7812-4386-a542-a6df76fe1fc0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:06.531820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:06.532949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:06.533501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:06.534356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] PATH_INFO: `/v3/users/f36a80d102cb463889bde77cb83acf15` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:06.536709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:06.540029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36a80d102cb463889bde77cb83acf15)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:06.549260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:06.617208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:06.618167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:06.618774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:06.619262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:06.621618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:06.624020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d066b83-a9f9-423c-8b40-babf5db077c1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:06.730255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 841/1663] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:06 2022] POST /identity/v3/users => generated 296 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:06.733973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:06.734485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:06.736870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:06.754733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36a80d102cb463889bde77cb83acf15'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:06.756045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36a80d102cb463889bde77cb83acf15'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:06.779937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] MFA Rules not processed for user `2c8ce3b0a82f47dc9ae6a4fb2abcaeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:06.782139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.791750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36a80d102cb463889bde77cb83acf15'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:06.811376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36a80d102cb463889bde77cb83acf15'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:06.905172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36a80d102cb463889bde77cb83acf15'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:06.913151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 823/1664] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:06 2022] DELETE /identity/v3/users/f36a80d102cb463889bde77cb83acf15 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:06.914191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:06.916930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.918806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 842/1665] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:06.920859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:06.925471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:06.964936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00266e0-2a11-4c35-9928-4076d3d20bfe None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.002185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:07.085212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c481-f536-4e04-a18f-2bf3a1920d21 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.095650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.096760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.097168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.097468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] PATH_INFO: `/v3/projects/4be885d3b724429fb72f06cd7750ecf1` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.099774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.099953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be885d3b724429fb72f06cd7750ecf1)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.100920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.101211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.101540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.101838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.102660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.108270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717d96f-91dc-454c-905f-e7e78b69d1e8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.126939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 843/1666] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:07.127567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.127863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.128142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.154282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:07.155926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.168634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 844/1667] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:07.168634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:07.170412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.253198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be885d3b724429fb72f06cd7750ecf1', 'request_id': 'req-66250385-c9b1-4d9a-9420-a74d725eb056'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:07.266587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be885d3b724429fb72f06cd7750ecf1', 'request_id': 'req-66250385-c9b1-4d9a-9420-a74d725eb056'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:07.279356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be885d3b724429fb72f06cd7750ecf1', 'request_id': 'req-66250385-c9b1-4d9a-9420-a74d725eb056'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:07.286519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387f189-01a9-456b-b0a2-f35b8746bfd9 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.373224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] The token cache is being invalidate because project 4be885d3b724429fb72f06cd7750ecf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:07.373824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:07.374790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:07.376887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 824/1668] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:06 2022] DELETE /identity/v3/projects/4be885d3b724429fb72f06cd7750ecf1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:07.377906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:07.379984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.418525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.420077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.420532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.420959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.426314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.536012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.538637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.543805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66250385-c9b1-4d9a-9420-a74d725eb056 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.549254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 845/1669] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:07.550509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:07.550684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.552488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.553677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.554114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.554581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] PATH_INFO: `/v3/projects/332707d531f5482d8802c1be0f10c288/users/90cb9b29525f4f7582635c9ed2add43d/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.586653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332707d531f5482d8802c1be0f10c288, user_id=90cb9b29525f4f7582635c9ed2add43d, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.590691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.671924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 825/1670] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:07 2022] PUT /identity/v3/projects/332707d531f5482d8802c1be0f10c288/users/90cb9b29525f4f7582635c9ed2add43d/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:07.673620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:07.676841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.700814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e517f-c9ce-411b-8a81-7da85bc6d122 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.822285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.822990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.823329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.823616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.826342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:07.884774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a67407-27b5-4af1-b6b1-a9336aa785c3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.903628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:07.905353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:07.910048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:07.910048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] PATH_INFO: `/v3/users/02b231b457af4b83bd7169e7afd88889` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:07.910048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b231b457af4b83bd7169e7afd88889)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.910048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.945176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:07.947921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:07.966694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 846/1671] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:07.967899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:07.982078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8114d-baed-48a3-86ab-6700fe497f30 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:07.999085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:08.000060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.000409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.000881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.005945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:08.005945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:08.016872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 847/1672] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:08.022781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:08.037402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19b8f0-39d1-4608-bba5-ef854f67cb4f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:08.044485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:08.046794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.047219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.048353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] PATH_INFO: `/v3/projects/332707d531f5482d8802c1be0f10c288/users/90cb9b29525f4f7582635c9ed2add43d/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.051598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332707d531f5482d8802c1be0f10c288, user_id=90cb9b29525f4f7582635c9ed2add43d, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:08.056172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13869d5-4a57-405e-94ea-eae7e7bfb246 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:08.083242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 848/1673] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:08 2022] PUT /identity/v3/projects/332707d531f5482d8802c1be0f10c288/users/90cb9b29525f4f7582635c9ed2add43d/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:08.090415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50c83-e49f-4130-97a9-fa42ffc8c218 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.091026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50c83-e49f-4130-97a9-fa42ffc8c218 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.092535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a50c83-e49f-4130-97a9-fa42ffc8c218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.112108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:08.153070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-29a50c83-e49f-4130-97a9-fa42ffc8c218 None None] MFA Rules not processed for user `90cb9b29525f4f7582635c9ed2add43d`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:08.154344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-29a50c83-e49f-4130-97a9-fa42ffc8c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:08.192192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b231b457af4b83bd7169e7afd88889'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:08.193327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b231b457af4b83bd7169e7afd88889'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:08.217172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b231b457af4b83bd7169e7afd88889'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:08.228015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b231b457af4b83bd7169e7afd88889'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:08.290919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b231b457af4b83bd7169e7afd88889'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:08.303440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 826/1674] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:07 2022] DELETE /identity/v3/users/02b231b457af4b83bd7169e7afd88889 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:08.358352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 849/1675] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:08.359677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:08.362567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:08.502374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:08.537306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7203a-b0bc-4c3d-bd9e-d548002e0181 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:08.551887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:08.553370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.553659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.556605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.576521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26d81-efa6-42a1-bed5-b79247347876 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.577156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26d81-efa6-42a1-bed5-b79247347876 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.577612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d26d81-efa6-42a1-bed5-b79247347876 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.579987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 850/1676] 10.222.0.59 () {54 vars in 912 bytes} [Thu Jul 14 11:17:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:08.586303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.586819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.587396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.675586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:08.677942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:08.718105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:08.719920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:08.747903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 827/1677] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:08.825638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 851/1678] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:08.869439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:08.872642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:08.914649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e913-c57d-47e6-8eb1-1a982c9bb75c service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:08.937486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:08.941901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:08.944719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:08.945909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:08.967828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:08.969331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:08.991582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 828/1679] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:09.043419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:09.045722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:09.076511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fe0ed-35b5-4092-8504-fb804e1dc757 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:09.081498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:09.082174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:09.082467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:09.082750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:09.085903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:09.193044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:09.195725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:09.213541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 852/1680] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:09.392138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:09.520273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:09.522581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:09.559105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:09.560606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:09.685189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b7d8-ffaa-49b5-83ef-deb178567533 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:09.704979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57265af8-fd1d-4e97-bbcd-24357d03aecf service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:09.706658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:09.707607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:09.707878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:09.708209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] PATH_INFO: `/v3/projects/d71ac221cd034943aad67230e8b108cb` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:09.710007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71ac221cd034943aad67230e8b108cb)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:09.710965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:09.711449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:09.711757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:09.712014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:09.712157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] PATH_INFO: `/v3/projects/1d6afac67e114813807a04a29e64dfa6` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:09.714344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6afac67e114813807a04a29e64dfa6)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:09.715530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: WARNING py.warnings [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:17:09.715530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: warnings.warn(msg) Jul 14 11:17:09.715530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: Jul 14 11:17:09.716652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:09.902697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71ac221cd034943aad67230e8b108cb', 'request_id': 'req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:09.911979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71ac221cd034943aad67230e8b108cb', 'request_id': 'req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:09.917916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6afac67e114813807a04a29e64dfa6', 'request_id': 'req-7965d9d0-0fb3-4601-9784-a57084e2c390'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:09.943871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6afac67e114813807a04a29e64dfa6', 'request_id': 'req-7965d9d0-0fb3-4601-9784-a57084e2c390'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:09.945954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71ac221cd034943aad67230e8b108cb', 'request_id': 'req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:09.984152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:10.027807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6afac67e114813807a04a29e64dfa6', 'request_id': 'req-7965d9d0-0fb3-4601-9784-a57084e2c390'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.080214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] The token cache is being invalidate because project 1d6afac67e114813807a04a29e64dfa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:10.080767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.090293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.097737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 853/1681] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:09 2022] DELETE /identity/v3/projects/1d6afac67e114813807a04a29e64dfa6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:10.102155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:10.111743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:10.127291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] The token cache is being invalidate because project d71ac221cd034943aad67230e8b108cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:10.127951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.129772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3ff1a1a4-9938-4782-80cf-dd4da12bfb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.135653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 829/1682] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:09 2022] DELETE /identity/v3/projects/d71ac221cd034943aad67230e8b108cb => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:10.137436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:10.137729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:10.138009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:10.231863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:10.233535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:10.359320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:10.381868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965d9d0-0fb3-4601-9784-a57084e2c390 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:10.429625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 830/1683] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:10.429625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:10.437411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:10.473795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da112441-e4e1-42cb-bc65-7bf1b072fc34 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:10.493087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:10.493994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:10.494289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:10.494571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] PATH_INFO: `/v3/users/ea9c05017b45471daa114c149cc848e3` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:10.497461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9c05017b45471daa114c149cc848e3)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:10.498463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:10.574288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:10.575244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:10.575536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:10.575818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] PATH_INFO: `/v3/users/788cf14f3ce440ec8ff3669e29a5fa4a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:10.579302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788cf14f3ce440ec8ff3669e29a5fa4a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:10.581341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:10.791315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9c05017b45471daa114c149cc848e3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.793560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9c05017b45471daa114c149cc848e3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.821003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9c05017b45471daa114c149cc848e3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.829148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9c05017b45471daa114c149cc848e3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.884442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788cf14f3ce440ec8ff3669e29a5fa4a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.901827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788cf14f3ce440ec8ff3669e29a5fa4a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.930914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788cf14f3ce440ec8ff3669e29a5fa4a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.932039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788cf14f3ce440ec8ff3669e29a5fa4a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.966113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-09bb40a6-99f8-4301-b374-cba9ce2f7cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9c05017b45471daa114c149cc848e3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:10.976869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 831/1684] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:10 2022] DELETE /identity/v3/users/ea9c05017b45471daa114c149cc848e3 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:10.981328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767960d-eea9-4341-957b-29a30b66711c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:10.981621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767960d-eea9-4341-957b-29a30b66711c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:10.981912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767960d-eea9-4341-957b-29a30b66711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.010175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4767960d-eea9-4341-957b-29a30b66711c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:11.011575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4767960d-eea9-4341-957b-29a30b66711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:11.054359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d362afb9-57ea-4d21-910b-8c5cf2c3c2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788cf14f3ce440ec8ff3669e29a5fa4a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:11.062009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 854/1685] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:10 2022] DELETE /identity/v3/users/788cf14f3ce440ec8ff3669e29a5fa4a => generated 0 bytes in 960 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:11.063704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5587-bd72-4a69-8b02-2d97f21efb71 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.064310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5587-bd72-4a69-8b02-2d97f21efb71 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.064588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5587-bd72-4a69-8b02-2d97f21efb71 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.067144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 855/1686] 10.222.0.59 () {54 vars in 960 bytes} [Thu Jul 14 11:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:11.073610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.074240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.074664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.134148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:11.135505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:11.156842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 832/1687] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:11.156842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.156842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.156842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.185507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:11.186919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:11.261340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 856/1688] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:11.263519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:11.277172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848d546-3b97-42c9-8d62-bada789f2b7b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:11.283362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:11.284591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.285077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.285558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.286620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:11.287182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:11.289130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:11.309892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 857/1689] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:11.311625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:11.325189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237958b-0c39-43b8-b620-aed2e5599a1a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:11.330967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:11.332260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.332879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.333798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.334946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:11.337893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:11.418306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 833/1690] 10.222.0.59 () {58 vars in 1057 bytes} [Thu Jul 14 11:17:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:11.436853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:11.462437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014144-e355-4b42-829e-8e7c57098818 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:11.474189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 858/1691] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:11 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:11.476358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:11.485183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:11.486019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.491427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.491427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.491427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:11.535461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:11.563188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:11.564337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:11.645475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 834/1692] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:11.661790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e76d70-83d1-449d-bef3-820504342bfd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:11.667341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:11.681709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:11.696839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:11.703864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:11.713997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:11.713997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:11.716082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:11.731215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:11.873681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357586af-0349-4b2a-b2fa-a4746d91af1e service placement] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:12.254001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 859/1693] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:11 2022] POST /identity/v3/users => generated 282 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:12.257688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:12.257688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.257688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.257688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.260623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:12.260623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:12.260623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 835/1694] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:12.265092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:12.265092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff89dc4-dfdf-49ae-ae27-ad698ce3729b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:12.265092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:12.265092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.269582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.269582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.269582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:12.269582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:12.310527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:12.318251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 860/1695] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:12.332435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dceed78-7c18-4b91-be47-cac01c1c533e service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:12.368180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:12.370092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.371511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.372246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.373449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:12.375862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e5847-1121-4a8a-862d-f01c182d7e93 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:12.401237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 836/1696] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:12.419917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:12.448288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d49aab-772b-464f-bf6c-341ecb845246 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:12.472818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:12.474106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.474106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.474968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] PATH_INFO: `/v3/projects/798f45c75a544c34b3e2a2afc40aa750/users/d20010648a5641e0a9e5fff4f0f49a13/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.524303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798f45c75a544c34b3e2a2afc40aa750, user_id=d20010648a5641e0a9e5fff4f0f49a13, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:12.534552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0968f-bdbe-4211-aece-6ec4fc6eeab7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:12.541639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.541927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.542375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.545796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:12.604781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] MFA Rules not processed for user `e3e8aa5fc932453e95080fa836dfac0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:12.606067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:12.639850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 861/1697] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:12 2022] PUT /identity/v3/projects/798f45c75a544c34b3e2a2afc40aa750/users/d20010648a5641e0a9e5fff4f0f49a13/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:12.658465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2aa2-341e-4655-b190-10ed1887c4a7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.658869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2aa2-341e-4655-b190-10ed1887c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.659295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2aa2-341e-4655-b190-10ed1887c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:12.778710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b1fe2aa2-341e-4655-b190-10ed1887c4a7 None None] MFA Rules not processed for user `d20010648a5641e0a9e5fff4f0f49a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:12.778710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b1fe2aa2-341e-4655-b190-10ed1887c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:12.879533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 837/1698] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:12.880573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:12.884661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:12.948092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 862/1699] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:12.948336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4db405-05dd-4702-ab05-f70195d4fa20 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:12.954343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347953af-85a2-4bf3-aead-28c4d1235944 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:12.954343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347953af-85a2-4bf3-aead-28c4d1235944 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:12.954343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347953af-85a2-4bf3-aead-28c4d1235944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.009923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:13.061272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-347953af-85a2-4bf3-aead-28c4d1235944 None None] MFA Rules not processed for user `3ac486b2d0ac42f9ba2fed7b17418b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:13.061973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-347953af-85a2-4bf3-aead-28c4d1235944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.120899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:13.121791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.122078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.122356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] PATH_INFO: `/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.124039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3c67aaebda4a3ea5e12a88d0d1063c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:13.132335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:13.272472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 863/1700] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:13.274149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.276218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.276529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.316020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] MFA Rules not processed for user `e3e8aa5fc932453e95080fa836dfac0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:13.317191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.354197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3c67aaebda4a3ea5e12a88d0d1063c', 'request_id': 'req-99566373-e5aa-4f61-b022-a3bc9fd82bdd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:13.372342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3c67aaebda4a3ea5e12a88d0d1063c', 'request_id': 'req-99566373-e5aa-4f61-b022-a3bc9fd82bdd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:13.383226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3c67aaebda4a3ea5e12a88d0d1063c', 'request_id': 'req-99566373-e5aa-4f61-b022-a3bc9fd82bdd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:13.436963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 864/1701] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:13.439902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:13.542552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] The token cache is being invalidate because project 3d3c67aaebda4a3ea5e12a88d0d1063c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:13.543069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:13.543884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:13.545305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0e57-5848-4a83-8164-13f55fb78f5f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:13.546209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 838/1702] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:12 2022] DELETE /identity/v3/projects/3d3c67aaebda4a3ea5e12a88d0d1063c => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:13.547286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:13.549060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.578937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:13.580157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.582570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.583285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.584865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:13.599623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:13.673783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99566373-e5aa-4f61-b022-a3bc9fd82bdd admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:13.681031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:13.682032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.682322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.682603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.684727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.736011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 865/1703] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:13 2022] POST /identity/v3/projects => generated 365 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:13.736011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:13.737292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.795951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:13.797514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:13.818157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 839/1704] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:13.819169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:13.821040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.839944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c78a9b-98c3-4254-b3e9-d06b112627a8 service nova] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:13.922430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb2bcd-36a5-40cc-802e-e24721a4eb8b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:13.934639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:13.935703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.936002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.936285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.939422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:13.940331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:13.940621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:13.940905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] PATH_INFO: `/v3/projects/6f2f4c8ea800491e9b6b4aff2dd75f4e` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:13.942849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2f4c8ea800491e9b6b4aff2dd75f4e)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:13.943624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:13.945354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.037926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.040121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e21f17-3fb9-409a-956a-7c5421ca8765 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.052499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 866/1705] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:14.053592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.053888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.054164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.100252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:14.101531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.102297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2f4c8ea800491e9b6b4aff2dd75f4e', 'request_id': 'req-719052a9-16fd-4da3-aec9-88455ac28ce6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.120646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2f4c8ea800491e9b6b4aff2dd75f4e', 'request_id': 'req-719052a9-16fd-4da3-aec9-88455ac28ce6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.134094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2f4c8ea800491e9b6b4aff2dd75f4e', 'request_id': 'req-719052a9-16fd-4da3-aec9-88455ac28ce6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.224089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 867/1706] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:14.224861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.232008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.281080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58251-ad66-4755-a750-20f7ee2201ca None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:14.283673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] The token cache is being invalidate because project 6f2f4c8ea800491e9b6b4aff2dd75f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:14.284257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.287965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-719052a9-16fd-4da3-aec9-88455ac28ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.293883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 840/1707] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:13 2022] DELETE /identity/v3/projects/6f2f4c8ea800491e9b6b4aff2dd75f4e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:14.295599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.295988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.296364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.310715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:14.313712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.314004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.314288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.321229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.323816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.351355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:14.354511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.370512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 841/1708] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:14.371187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.376941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.391903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645cd5d-4a57-4ca2-8b22-eb8c4162ec60 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:14.398916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:14.400661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.401217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.401483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] PATH_INFO: `/v3/users/90cb9b29525f4f7582635c9ed2add43d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.402872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cb9b29525f4f7582635c9ed2add43d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.405051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.445010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 868/1709] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:14 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:14.445010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.461166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a42358-4ab7-4450-9e8b-9d994d6ffbe1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:14.466995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:14.467960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.468345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.468727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.480091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:14.492765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.492765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.517772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 869/1710] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:14.519275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.523797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.554636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244026-0bda-4dea-ae47-deccce02f0b2 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:14.564355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:14.564355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.564355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.564355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.565474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.565899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.591981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 870/1711] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:14.592962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.636683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49654369-0e95-4eb5-be5f-276ca090acff admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:14.643424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cb9b29525f4f7582635c9ed2add43d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.647222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cb9b29525f4f7582635c9ed2add43d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.657193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:14.661817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:14.661817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:14.661817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:14.663669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:14.672243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:14.684584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cb9b29525f4f7582635c9ed2add43d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.697848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cb9b29525f4f7582635c9ed2add43d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.737643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:14.820687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cb9b29525f4f7582635c9ed2add43d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:14.840864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 842/1712] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:14 2022] DELETE /identity/v3/users/90cb9b29525f4f7582635c9ed2add43d => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:14.842020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.847502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:14.859010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 871/1713] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:14 2022] POST /identity/v3/projects => generated 365 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:14.862663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:14.885129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:15.095618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:15.122155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1af34-4502-401b-8a4f-88d60a12bf1d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.205600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2327c-ea66-47fa-9bf6-357b59848092 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.212243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.214094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.214527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.215053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.216254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:15.219632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709344db-ffd1-4cec-8fb8-9b0f80c952f4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:15.369904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 872/1714] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:14 2022] POST /identity/v3/users => generated 283 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:15.377915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e83c8-59ea-4b3d-a189-60154b304c81 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.377915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e83c8-59ea-4b3d-a189-60154b304c81 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.381234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e83c8-59ea-4b3d-a189-60154b304c81 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.399706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 873/1715] 10.222.0.59 () {54 vars in 912 bytes} [Thu Jul 14 11:17:15 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:15.405227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e83c8-59ea-4b3d-a189-60154b304c81 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:15.408613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.413093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.413993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.413993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] PATH_INFO: `/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9/users/8ff3f4f43a7847feb7eb73c31fed8034/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.445627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e83c8-59ea-4b3d-a189-60154b304c81 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.445914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ebd2a48f9442aaad959d08cf5a42c9, user_id=8ff3f4f43a7847feb7eb73c31fed8034, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:15.453204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:15.454632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.456840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.457472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.458597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.460247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:15.462467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f44c16-ab3f-4d57-9a38-f24980f3c662 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:15.509280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 874/1716] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:15.511510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.514579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.514985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.566830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:15.571649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:15.575697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 843/1717] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:14 2022] PUT /identity/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9/users/8ff3f4f43a7847feb7eb73c31fed8034/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:15.576864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:15.747567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 875/1718] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:17:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:15.748682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:15.792843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3304f-e0b3-4f2e-896d-e241a5c70f23 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.803181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.804218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.804600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.804974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.805831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:15.807922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:15.830387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 844/1719] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:15.831378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:15.835434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:15.861896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8dd39-4392-45cc-92f7-3cf99a85b64f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.870522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.871599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.872005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.872389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:15.882546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:15.885990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:15.899626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 845/1720] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:15.900732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:15.943579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b08624-35c5-46ef-bbc0-345c4c77a1e5 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:15.953685 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:15.955065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:15.955504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:15.955880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] PATH_INFO: `/v3/projects/c0796a0fc9004fdd9ac570976fcf3372/users/c21d80ed4ae544cd9e565915390cd945/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.002464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0796a0fc9004fdd9ac570976fcf3372, user_id=c21d80ed4ae544cd9e565915390cd945, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:16.007305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbdcde-2c74-400f-bafb-05b6b2298d26 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:16.029865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590889b-de72-4453-abf4-8e6973b110bb None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:16.053560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:16.058562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:16.058781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:16.064746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.067643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:16.102033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:16.133516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 846/1721] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:15 2022] PUT /identity/v3/projects/c0796a0fc9004fdd9ac570976fcf3372/users/c21d80ed4ae544cd9e565915390cd945/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:16.140054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:16.140509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:16.140819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.164532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 876/1722] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:16.165686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:16.222880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838e303-1490-4f9c-9b03-35cbf000ee94 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:16.225843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] MFA Rules not processed for user `c21d80ed4ae544cd9e565915390cd945`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:16.227309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:16.389574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 847/1723] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:16.393283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:16.395845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:16.520021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:16.524071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:16.525761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:16.526106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] PATH_INFO: `/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9/users/8ff3f4f43a7847feb7eb73c31fed8034/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.535053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ebd2a48f9442aaad959d08cf5a42c9, user_id=8ff3f4f43a7847feb7eb73c31fed8034, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:16.550650 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:16.604791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 877/1724] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:16 2022] PUT /identity/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9/users/8ff3f4f43a7847feb7eb73c31fed8034/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:16.605821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:16.649886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210e14d-3389-464f-92b4-a53dd63cf7b5 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:16.667368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:16.784171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:16.785183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:16.786501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:16.786501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] PATH_INFO: `/v3/projects/332707d531f5482d8802c1be0f10c288` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.788186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=332707d531f5482d8802c1be0f10c288)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:16.790421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:16.854218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01ac4-251c-4027-8077-c784786740d4 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:16.859999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:16.860774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:16.861222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:16.861595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:16.862409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:16.864852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b745a-8769-448e-8de0-a3fa47e28b19 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:16.907815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:16.966801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332707d531f5482d8802c1be0f10c288', 'request_id': 'req-b67c8d39-868f-410c-b35f-b3172487ad48'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:16.976793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332707d531f5482d8802c1be0f10c288', 'request_id': 'req-b67c8d39-868f-410c-b35f-b3172487ad48'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:16.999548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 878/1725] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:16 2022] POST /identity/v3/projects => generated 363 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:17.001718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:17.002719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:17.006830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:17.022707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332707d531f5482d8802c1be0f10c288', 'request_id': 'req-b67c8d39-868f-410c-b35f-b3172487ad48'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:17.093907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] MFA Rules not processed for user `8ff3f4f43a7847feb7eb73c31fed8034`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:17.095581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:17.190518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] The token cache is being invalidate because project 332707d531f5482d8802c1be0f10c288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:17.190929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:17.191545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:17.195761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 848/1726] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:16 2022] DELETE /identity/v3/projects/332707d531f5482d8802c1be0f10c288 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:17.201574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:17.205070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:17.411871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 879/1727] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:17.412483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:17.428931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:17.457304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8d39-868f-410c-b35f-b3172487ad48 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:17.667892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bb065-e314-4176-a432-e0a05fa64483 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:17.671432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:17.674944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:17.674944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:17.674944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] PATH_INFO: `/v3/users/2c12aa6fbe0f4ff2af596fff374e423b` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:17.674944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c12aa6fbe0f4ff2af596fff374e423b)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:17.677902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:17.697995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:17.703820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:17.703820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:17.703820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:17.703820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:17.703820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004adc75-4c67-4780-9370-a7a30f4411ec admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:17.757334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:17.825083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 880/1728] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:17 2022] POST /identity/v3/users => generated 288 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:17.830912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a1817-4d29-46ec-9389-58dcb0aa0da4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:17.833394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a1817-4d29-46ec-9389-58dcb0aa0da4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:17.835520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a1817-4d29-46ec-9389-58dcb0aa0da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:17.874894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-371a1817-4d29-46ec-9389-58dcb0aa0da4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:17.877854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-371a1817-4d29-46ec-9389-58dcb0aa0da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:17.958135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 881/1729] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:17.962025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c12aa6fbe0f4ff2af596fff374e423b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:17.962324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:17.963856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:17.965735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c12aa6fbe0f4ff2af596fff374e423b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:17.977692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.022841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c12aa6fbe0f4ff2af596fff374e423b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:18.043940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] MFA Rules not processed for user `e3e8aa5fc932453e95080fa836dfac0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:18.047522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.067385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c12aa6fbe0f4ff2af596fff374e423b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:18.156811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c12aa6fbe0f4ff2af596fff374e423b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:18.165494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 849/1730] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:17 2022] DELETE /identity/v3/users/2c12aa6fbe0f4ff2af596fff374e423b => generated 0 bytes in 969 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:18.166638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:18.168705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.205642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:18.233429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 882/1731] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:18.234854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:18.237153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.288236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e1a72-0bc3-42d9-9996-15d98df1bf07 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:18.388541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:18.398413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2127e85-0803-4f37-ae02-186c4bae730a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:18.422682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:18.423707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.424175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.424629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.429103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.433469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:18.434163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.434452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.434733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.435505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:18.437130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e60fb-9a77-4a8a-9ce6-59ff7e2001e5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:18.464874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 883/1732] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:18.464874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1519-d1ca-449b-813e-6c4b39247d38 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.464874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1519-d1ca-449b-813e-6c4b39247d38 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.464874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea1519-d1ca-449b-813e-6c4b39247d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.509652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-30ea1519-d1ca-449b-813e-6c4b39247d38 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:18.510897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-30ea1519-d1ca-449b-813e-6c4b39247d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.531142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 884/1733] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:18.534590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.534881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.537357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.642947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:18.644580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ee96-b262-4dc8-a226-fd772ff872a1 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:18.667727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 850/1734] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:18.669367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.669832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.670287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.710634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:18.715283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.728857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 851/1735] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:18.729793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:18.758241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:18.759563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:18.762862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae844d-456c-4209-9d2d-4188dae3c0c5 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:18.780048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:18.782668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:18.783580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:18.784567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] PATH_INFO: `/v3/projects/a04a881f219142a28772322f50e9716b/users/e342f32cc28a4c2ea58562b1e1bb512a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:18.837641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04a881f219142a28772322f50e9716b, user_id=e342f32cc28a4c2ea58562b1e1bb512a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:18.852385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:18.969760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 852/1736] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:18 2022] PUT /identity/v3/projects/a04a881f219142a28772322f50e9716b/users/e342f32cc28a4c2ea58562b1e1bb512a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:18.976810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.034132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 885/1737] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:19.035032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.045799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:19.150570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53788-d2fd-4d26-9faf-081040d1780e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.185234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:19.201157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49601a36-90fc-4be2-b771-8de3b5f633ed None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.215479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.216274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.216632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.216974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.217811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:19.219235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.220194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.251574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 853/1738] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:19.255857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.286697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a26f6-4325-45fc-9903-e7fc8b053331 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.328544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.329906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.332720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.333475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.334978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.348719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.358708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:19.394065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 854/1739] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:19.403030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.442463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.443400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.443690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.451494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.493046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.501040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.554784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 886/1740] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:19.556004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.601404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ec17c-f610-42af-9424-e0b5226a4ab6 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.616132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.616278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e873-f73e-467d-a07b-b25958e34290 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.616882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.622385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.625488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.631727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.631727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.631996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.631996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.631996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.633138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:19.635094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.787399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:19.796838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.798466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.808228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 887/1741] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:19 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:19.808228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.812894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 855/1742] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:19.818665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:19.836638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:19.966111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62ee5-a84d-423d-8299-f2fa5da5b9c4 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:19.974509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:19.976218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:19.976769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:19.977214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] PATH_INFO: `/v3/projects/a04a881f219142a28772322f50e9716b/users/e342f32cc28a4c2ea58562b1e1bb512a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:19.980744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04a881f219142a28772322f50e9716b, user_id=e342f32cc28a4c2ea58562b1e1bb512a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:19.984409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:19.999981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 888/1743] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:19 2022] PUT /identity/v3/projects/a04a881f219142a28772322f50e9716b/users/e342f32cc28a4c2ea58562b1e1bb512a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:20.001248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:20.037892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:20.122287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7a27-712f-4aee-86d9-fa90043cc6cd service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:20.270482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:20.271923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.272352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.272792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.279156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017acb5-b6de-466e-ae83-33483267c22d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:20.287521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:20.288389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.288949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.289320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.290625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:20.293463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:20.406053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:20.406053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3841f9-a597-480a-9da8-74b2b405b4a9 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:20.431540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 856/1744] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:17:20.442384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b5ae-f42b-4892-a496-f5b5cf23b8e3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.451114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b5ae-f42b-4892-a496-f5b5cf23b8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.451457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9b5ae-f42b-4892-a496-f5b5cf23b8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.462676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 889/1745] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:20 2022] POST /identity/v3/users => generated 294 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:20.465677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:20.491006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec4fa5-7f33-4d88-a540-06bbfc13fb36 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:20.526281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:20.527149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.527443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.527723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.533611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:20.554892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:20.556708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7eb9b5ae-f42b-4892-a496-f5b5cf23b8e3 None None] MFA Rules not processed for user `e342f32cc28a4c2ea58562b1e1bb512a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:20.558200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7eb9b5ae-f42b-4892-a496-f5b5cf23b8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:20.626218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 890/1746] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:20.628474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:20.674028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16068ac9-c9b2-47df-a69f-f611b5ae2033 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:20.703835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:20.707350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.708213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.709600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.711698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:20.727892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:20.776290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 891/1747] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:20.781788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:20.867912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5715-5d5a-4e66-9a06-eba39e161f0b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:20.891046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:20.892434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:20.894137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:20.897114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] PATH_INFO: `/v3/projects/6a81e77237f6409aa06e17f4fb7107e0/users/3b20c71527e84eb9842989f2fbf67e5f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:20.934788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:20.971956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a81e77237f6409aa06e17f4fb7107e0, user_id=3b20c71527e84eb9842989f2fbf67e5f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:20.988001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:21.023416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 857/1748] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 591 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:21.038318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac2af-72b3-40ff-a727-858121be8458 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:21.038822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac2af-72b3-40ff-a727-858121be8458 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:21.039276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ac2af-72b3-40ff-a727-858121be8458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:21.086971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 892/1749] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:20 2022] PUT /identity/v3/projects/6a81e77237f6409aa06e17f4fb7107e0/users/3b20c71527e84eb9842989f2fbf67e5f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:21.088350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:21.104409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:21.142316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3c0ac2af-72b3-40ff-a727-858121be8458 None None] MFA Rules not processed for user `3ac486b2d0ac42f9ba2fed7b17418b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:21.148704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3c0ac2af-72b3-40ff-a727-858121be8458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:21.418335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 858/1750] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:21.418335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:21.419050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:21.419050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:21.420342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:21.444704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f5906-6908-493f-bb9e-0cf3c6898f0f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:21.454817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:21.455678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:21.456017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:21.456352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:21.525455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] MFA Rules not processed for user `3b20c71527e84eb9842989f2fbf67e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:21.526718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:21.655321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:21.655597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:21.655597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:21.698295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 893/1751] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:21.698295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:21.698295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:21.739490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 859/1752] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:21.743907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:21.835015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c85a8-2163-4d87-820b-dd6df0025482 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:21.843836 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:21.844829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:21.845308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:21.845940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:21.847058 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:21.851156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:21.965018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 860/1753] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:21 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:21.971902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:22.003604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d984-b48d-41ff-b32e-fe5bc86e7e61 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:22.107458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f99b-1bfd-4e9c-aee2-68abc0f7f706 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:22.123461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:22.123461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.123461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.123461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.123937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:22.123937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:22.153745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:22.153745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.153745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.153745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] PATH_INFO: `/v3/users/d20010648a5641e0a9e5fff4f0f49a13` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.161300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20010648a5641e0a9e5fff4f0f49a13)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:22.164449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:22.234117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 861/1754] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:21 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:22.234994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:22.238267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:22.264633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:22.289881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ab2f6-2c9d-4e62-a005-8fe50956289a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:22.304631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:22.306940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.307563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.307813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] PATH_INFO: `/v3/projects/0ac8b98f65494486a1b109c0bc4b9b52` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.314628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac8b98f65494486a1b109c0bc4b9b52)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:22.349770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:22.353675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20010648a5641e0a9e5fff4f0f49a13'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.367026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20010648a5641e0a9e5fff4f0f49a13'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.420790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20010648a5641e0a9e5fff4f0f49a13'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.424087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20010648a5641e0a9e5fff4f0f49a13'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.458193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:22.602139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20010648a5641e0a9e5fff4f0f49a13'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.617013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 894/1755] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:21 2022] DELETE /identity/v3/users/d20010648a5641e0a9e5fff4f0f49a13 => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:22.617273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:22.624288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:22.719735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac8b98f65494486a1b109c0bc4b9b52', 'request_id': 'req-dd19838a-33e8-417b-aa0f-d94f8800a268'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.732679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac8b98f65494486a1b109c0bc4b9b52', 'request_id': 'req-dd19838a-33e8-417b-aa0f-d94f8800a268'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.776234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac8b98f65494486a1b109c0bc4b9b52', 'request_id': 'req-dd19838a-33e8-417b-aa0f-d94f8800a268'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.819223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99f646-0af4-49f7-87b0-781a1423a2a0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:22.905529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] The token cache is being invalidate because project 0ac8b98f65494486a1b109c0bc4b9b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dd19838a-33e8-417b-aa0f-d94f8800a268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 862/1756] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:22 2022] DELETE /identity/v3/projects/0ac8b98f65494486a1b109c0bc4b9b52 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.922458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.933284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:22.937830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.937830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.938084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.938657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:22.951723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9503c7-bed0-4668-975d-fbbd89c7f594 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:22.985454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 895/1757] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:22.987761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccac54-e9ae-467d-9f8c-349b2b43482e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:22.987917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccac54-e9ae-467d-9f8c-349b2b43482e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:22.988409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccac54-e9ae-467d-9f8c-349b2b43482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:22.991515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:22.993029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.036165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-10ccac54-e9ae-467d-9f8c-349b2b43482e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:23.037608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-10ccac54-e9ae-467d-9f8c-349b2b43482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.059386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 896/1758] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:23.061938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.062408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.062824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.104513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:23.105726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.122857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 863/1759] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jul 14 11:17:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:23.124907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.128539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 897/1760] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:23.128860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.131341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.134000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.153129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:23.162025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426e941-aecb-42a4-a0f1-2bee9c3defda None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.165251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37cc10-e565-4e7a-bab0-5595578d23c8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.180019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.180725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.181025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.181305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.186757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.187783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.188075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.188358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] PATH_INFO: `/v3/projects/ee43ff71ee1e4be39168efec8121098f/users/4f40cd02de374369b055214214ff97d6/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.220863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.225783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb42398-e3f4-4c54-a807-da4e44073c39 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.234243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee43ff71ee1e4be39168efec8121098f, user_id=4f40cd02de374369b055214214ff97d6, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.239848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.251640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 898/1761] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:23.254096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.254504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.255290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.279369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:23.281595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.297600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 899/1762] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:23.297600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.305414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.372518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19156ab8-2c20-406b-b466-740c7985fb17 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.393706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.394671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.406711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.407076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.411259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 864/1763] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:23 2022] PUT /identity/v3/projects/ee43ff71ee1e4be39168efec8121098f/users/4f40cd02de374369b055214214ff97d6/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:23.412377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.432893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.432893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.462353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 900/1764] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:23.463631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.536903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9615c8-e098-4a2d-8144-ec90f0704bef service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.623349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d0456-32f0-4bd1-8d39-feee1eea9929 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.645845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.646733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.648988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.649457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.650250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:23.650569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.654495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.705298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 865/1765] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:23.709057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.713728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:23.744397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.745987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.748178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.748178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.748656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.752914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.758859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4733fa-92ee-4312-a24f-44ed3b0fdb8c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.776473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.782762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.782762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.782762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] PATH_INFO: `/v3/users/c21d80ed4ae544cd9e565915390cd945` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.782762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21d80ed4ae544cd9e565915390cd945)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.784648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:23.823430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 901/1766] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:23.823430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:23.829790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ab57-cb34-4f81-97ad-3cc464390d50 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:23.840013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:23.841971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:23.843255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:23.844091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:23.845971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:23.853361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:24.018093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 902/1767] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:23 2022] POST /identity/v3/projects => generated 405 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:24.018093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:24.071903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21d80ed4ae544cd9e565915390cd945'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:24.073555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21d80ed4ae544cd9e565915390cd945'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:24.112402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21d80ed4ae544cd9e565915390cd945'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:24.127984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:24.134708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21d80ed4ae544cd9e565915390cd945'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:24.183239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae29c85-b8ea-4276-b40a-1dfae82d1dce admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:24.190118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:24.194553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.194991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.197665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] PATH_INFO: `/v3/projects/ee43ff71ee1e4be39168efec8121098f/users/4f40cd02de374369b055214214ff97d6/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.204328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee43ff71ee1e4be39168efec8121098f, user_id=4f40cd02de374369b055214214ff97d6, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:24.208474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:24.233006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 903/1768] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:24 2022] PUT /identity/v3/projects/ee43ff71ee1e4be39168efec8121098f/users/4f40cd02de374369b055214214ff97d6/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:24.233006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:24.233006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:24.276675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0c504766-42e2-4f62-ad8e-7b7b921755ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21d80ed4ae544cd9e565915390cd945'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:24.288522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 866/1769] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:23 2022] DELETE /identity/v3/users/c21d80ed4ae544cd9e565915390cd945 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:24.288522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476ecdd-92ae-4afa-af98-6fd0a000e24a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.288522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476ecdd-92ae-4afa-af98-6fd0a000e24a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.289412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476ecdd-92ae-4afa-af98-6fd0a000e24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.381977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6476ecdd-92ae-4afa-af98-6fd0a000e24a None None] MFA Rules not processed for user `4f40cd02de374369b055214214ff97d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:24.383533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6476ecdd-92ae-4afa-af98-6fd0a000e24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:24.386099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27daee-d4ba-4f88-b7c4-f45bf2a1ce97 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:24.528192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:24.592435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:24.593619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.597367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.597367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.597367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:24.612650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 867/1770] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:24.614206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:24.626104 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.626651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.626651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.691341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:24.692697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:24.714689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 868/1771] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:24.715897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:24.724339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:24.784646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 904/1772] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:24 2022] POST /identity/v3/users => generated 303 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:24.785902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:24.793283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:24.877800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4614567-69f4-4828-8f80-0901b7bb7e46 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:24.888571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:24.889306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.889599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.889882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.927250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:24.928747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:24.947671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 869/1773] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:24.954598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:24.985852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0ad3-355c-4c1c-848d-2048be45ada8 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:24.987229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b65c0-3f9b-44d5-93f5-80bacf331a47 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:24.995621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:24.996726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:24.997026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:24.997308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:24.998246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:25.004007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:25.014507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:25.015476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:25.016256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:25.016761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:25.031763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:25.034039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:25.042028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 870/1774] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:25.055757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:25.064621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:25.079949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 905/1775] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:25.084124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:25.087762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:25.088291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:25.099334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a46d73-3857-4215-b243-b1fd15c25159 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:25.120583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:25.121602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:25.121919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:25.122216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] PATH_INFO: `/v3/users/3d63b01c29064fb98655a558b5cd527f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:25.126956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d63b01c29064fb98655a558b5cd527f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:25.129202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:25.139768 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88f47-c970-4133-a7bc-0ca6b41e3e10 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:25.150020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:25.153939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:25.153939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:25.153939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] PATH_INFO: `/v3/users/3b20c71527e84eb9842989f2fbf67e5f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:25.154619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b20c71527e84eb9842989f2fbf67e5f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:25.156327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:25.483896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d63b01c29064fb98655a558b5cd527f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.487269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d63b01c29064fb98655a558b5cd527f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.520534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d63b01c29064fb98655a558b5cd527f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.537947 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d63b01c29064fb98655a558b5cd527f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.770946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b20c71527e84eb9842989f2fbf67e5f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.793285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b20c71527e84eb9842989f2fbf67e5f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.814443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b20c71527e84eb9842989f2fbf67e5f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.815313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b20c71527e84eb9842989f2fbf67e5f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.819007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:25.824779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d63b01c29064fb98655a558b5cd527f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:25.833285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 871/1776] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:25 2022] DELETE /identity/v3/users/3d63b01c29064fb98655a558b5cd527f => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:25.834296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:25.839742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.008072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d9d76-7d18-4709-9a88-fa93557a5273 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:26.016085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d52c2f4-28fa-4511-8108-0730c04d0ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b20c71527e84eb9842989f2fbf67e5f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.034721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 906/1777] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:25 2022] DELETE /identity/v3/users/3b20c71527e84eb9842989f2fbf67e5f => generated 0 bytes in 954 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:26.042541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694600-7a62-445c-9031-a1687906b4c0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:26.042907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694600-7a62-445c-9031-a1687906b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:26.046058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694600-7a62-445c-9031-a1687906b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:26.056695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:26.058970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:26.058970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:26.066501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:26.068925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:26.072888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:26.079686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:26.106036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 872/1778] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:26.106036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:26.108150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.126438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5c694600-7a62-445c-9031-a1687906b4c0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:26.130263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5c694600-7a62-445c-9031-a1687906b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.134906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9c1c6-4c8c-412f-b0d8-87d5dce5ed79 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:26.243449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:26.319081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:26.320064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:26.320478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:26.320752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] PATH_INFO: `/v3/projects/798f45c75a544c34b3e2a2afc40aa750` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:26.325665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798f45c75a544c34b3e2a2afc40aa750)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:26.328506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:26.396485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 907/1779] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:26.396485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:26.396485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:26.396485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:26.405896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:26.407533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.418038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 908/1780] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:26.424797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:26.424797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.455648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8185-dc0a-4bf9-81eb-b99e8bccd36d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:26.483238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:26.484209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:26.484507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:26.484780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] PATH_INFO: `/v3/projects/586d9118a6c647ff91f9ab80efaee9a3/users/bc4c0fe4d5794e0590060294fcb8964a/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:26.528506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] RBAC: Authorizing `identity:create_grant(project_id=586d9118a6c647ff91f9ab80efaee9a3, user_id=bc4c0fe4d5794e0590060294fcb8964a, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:26.538119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:26.650742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 909/1781] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:26 2022] PUT /identity/v3/projects/586d9118a6c647ff91f9ab80efaee9a3/users/bc4c0fe4d5794e0590060294fcb8964a/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:26.653454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:26.657032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:26.675689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798f45c75a544c34b3e2a2afc40aa750', 'request_id': 'req-ccd7fc7e-3969-4a59-bace-6254685481a6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.696165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798f45c75a544c34b3e2a2afc40aa750', 'request_id': 'req-ccd7fc7e-3969-4a59-bace-6254685481a6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.718886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798f45c75a544c34b3e2a2afc40aa750', 'request_id': 'req-ccd7fc7e-3969-4a59-bace-6254685481a6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.826504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:26.840540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0ad44-1a1b-440e-81e2-c6525bc7359a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:26.843909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] The token cache is being invalidate because project 798f45c75a544c34b3e2a2afc40aa750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:26.844689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.847401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:26.858701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 873/1782] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:26 2022] DELETE /identity/v3/projects/798f45c75a544c34b3e2a2afc40aa750 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:26.860158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:26.862785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.037138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:27.038103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.038484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.038877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.041214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.041642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7fc7e-3969-4a59-bace-6254685481a6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:27.056008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:27.058019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.058318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.059103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.061513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.158368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:27.168646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:27.170339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086864d9-1954-403e-86ea-b9078eea686f service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:27.195456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 910/1783] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:27.196821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823d23-b5ea-4c3f-bc96-9c0da1f8d63f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.197116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823d23-b5ea-4c3f-bc96-9c0da1f8d63f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.197468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03823d23-b5ea-4c3f-bc96-9c0da1f8d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.242772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-03823d23-b5ea-4c3f-bc96-9c0da1f8d63f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:27.246567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-03823d23-b5ea-4c3f-bc96-9c0da1f8d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.266936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:27.270901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ca36-526e-4aaa-83bc-4f35c1b859ae service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:27.274568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 911/1784] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:27.276789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.287665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.287665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.287665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 912/1785] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:17:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:27.287665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:27.288000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 874/1786] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:27.288772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.306840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.307171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.307448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.315309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d11423-25a1-4da0-81c8-4db6887e0e99 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:27.329153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:27.330291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.330663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.331010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] PATH_INFO: `/v3/users/8ff3f4f43a7847feb7eb73c31fed8034` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.332789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff3f4f43a7847feb7eb73c31fed8034)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:27.336172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:27.354943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:27.360189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.570500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 875/1787] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:27.571944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:27.584068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eafd6-a6bc-4d4b-bd65-2ac8dd53af40 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:27.590039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:27.590039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.590039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.590039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.604305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:27.606332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:27.637414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 876/1788] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:27.639095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:27.640723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.645271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff3f4f43a7847feb7eb73c31fed8034'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.658324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a0b02-e55a-4ab6-9035-143b7364b702 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:27.659389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff3f4f43a7847feb7eb73c31fed8034'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.665367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:27.666264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:27.666938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:27.667484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] PATH_INFO: `/v3/projects/c0796a0fc9004fdd9ac570976fcf3372` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:27.669257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0796a0fc9004fdd9ac570976fcf3372)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:27.671375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:27.694856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff3f4f43a7847feb7eb73c31fed8034'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.698359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff3f4f43a7847feb7eb73c31fed8034'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.804318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff3f4f43a7847feb7eb73c31fed8034'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.843918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 913/1789] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:27 2022] DELETE /identity/v3/users/8ff3f4f43a7847feb7eb73c31fed8034 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:27.855402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:27.863434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:27.876439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0796a0fc9004fdd9ac570976fcf3372', 'request_id': 'req-a1b958d8-c708-4628-92f3-de4b21ebf754'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.896551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0796a0fc9004fdd9ac570976fcf3372', 'request_id': 'req-a1b958d8-c708-4628-92f3-de4b21ebf754'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:27.911092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0796a0fc9004fdd9ac570976fcf3372', 'request_id': 'req-a1b958d8-c708-4628-92f3-de4b21ebf754'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.019350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] The token cache is being invalidate because project c0796a0fc9004fdd9ac570976fcf3372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:28.021623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.023634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.028972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 877/1790] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:27 2022] DELETE /identity/v3/projects/c0796a0fc9004fdd9ac570976fcf3372 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:28.032949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:28.040544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:28.047157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ce168-eecc-4bed-97d5-dfdc8098b6e8 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:28.059361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:28.060112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:28.060462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:28.060740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:28.064190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:28.093952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:28.097628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:28.135333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 914/1791] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:28.198015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:28.228371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b958d8-c708-4628-92f3-de4b21ebf754 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:28.353848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:28.354927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:28.358489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:28.358927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] PATH_INFO: `/v3/users/e342f32cc28a4c2ea58562b1e1bb512a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:28.360535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e342f32cc28a4c2ea58562b1e1bb512a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:28.365762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:28.712634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:28.712634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:28.713062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e342f32cc28a4c2ea58562b1e1bb512a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.713062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e342f32cc28a4c2ea58562b1e1bb512a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.738016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e342f32cc28a4c2ea58562b1e1bb512a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.746622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251b34-515d-405a-a855-83a6c5b199b8 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:28.748410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e342f32cc28a4c2ea58562b1e1bb512a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.756800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:28.757887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:28.758451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:28.758799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] PATH_INFO: `/v3/projects/cc04e272fd2e4559bc35a25ed1252c23` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:28.760542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc04e272fd2e4559bc35a25ed1252c23)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:28.762647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:28.804034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:28.888484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e342f32cc28a4c2ea58562b1e1bb512a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:28.898313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 878/1792] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:28 2022] DELETE /identity/v3/users/e342f32cc28a4c2ea58562b1e1bb512a => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:28.899315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:28.907064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.024523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc04e272fd2e4559bc35a25ed1252c23', 'request_id': 'req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.041981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc04e272fd2e4559bc35a25ed1252c23', 'request_id': 'req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.064261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71345e8f-a08f-4e63-93e0-0eac1044c945 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:29.138958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc04e272fd2e4559bc35a25ed1252c23', 'request_id': 'req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.161675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] The token cache is being invalidate because project cc04e272fd2e4559bc35a25ed1252c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:29.161978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.168604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8b35daa0-1ca2-4e16-bc33-78c2b1f84a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.174726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 915/1793] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:28 2022] DELETE /identity/v3/projects/cc04e272fd2e4559bc35a25ed1252c23 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:29.184120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80ee9-50dd-49a7-8128-5c2cf5bd5990 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.184418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80ee9-50dd-49a7-8128-5c2cf5bd5990 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.184750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a80ee9-50dd-49a7-8128-5c2cf5bd5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.194632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:29.196670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.197193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.197766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] PATH_INFO: `/v3/projects/6a81e77237f6409aa06e17f4fb7107e0` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.200153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a81e77237f6409aa06e17f4fb7107e0)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:29.202465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:29.260189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-91a80ee9-50dd-49a7-8128-5c2cf5bd5990 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:29.261571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-91a80ee9-50dd-49a7-8128-5c2cf5bd5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.303893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 916/1794] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:29.317770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.318301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.318724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.365747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:29.367226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.385972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a81e77237f6409aa06e17f4fb7107e0', 'request_id': 'req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.400145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a81e77237f6409aa06e17f4fb7107e0', 'request_id': 'req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.439349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a81e77237f6409aa06e17f4fb7107e0', 'request_id': 'req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.485261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 917/1795] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:29.486639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:29.522414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf5663-256d-429e-b8e1-dafaf663aac0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:29.531252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:29.531947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.532240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.532507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.533256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:29.533550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:29.537436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:29.562542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] The token cache is being invalidate because project 6a81e77237f6409aa06e17f4fb7107e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:29.563036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.563885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:29.571868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 879/1796] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:28 2022] DELETE /identity/v3/projects/6a81e77237f6409aa06e17f4fb7107e0 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:29.574782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 918/1797] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:29.576094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:29.576534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:29.577906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.578908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.595166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9c9ed-2a99-4133-b3d5-a73dd5a12750 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:29.615317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:29.616009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.616296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.616592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.617282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:29.619235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:29.623163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad4bda-b2f8-4c5c-a38f-586dc030dcfe admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:29.641473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:29.646008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.656625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.657164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] PATH_INFO: `/v3/users/4f40cd02de374369b055214214ff97d6` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.663372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f40cd02de374369b055214214ff97d6)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:29.674700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:29.834305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 919/1798] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:29 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:29.836894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:29.843396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:29.970146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f9632-dda1-4577-9baa-62e9cdce86b1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:29.991232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:29.996370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:29.997838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:29.997838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:29.997838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] PATH_INFO: `/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:29.999656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ebd2a48f9442aaad959d08cf5a42c9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:30.002433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:30.043962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f40cd02de374369b055214214ff97d6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.045160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f40cd02de374369b055214214ff97d6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.073887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f40cd02de374369b055214214ff97d6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.091446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f40cd02de374369b055214214ff97d6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.225110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ebd2a48f9442aaad959d08cf5a42c9', 'request_id': 'req-8051f206-1cb8-4662-bf57-07814172392a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.225689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f40cd02de374369b055214214ff97d6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.235074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ebd2a48f9442aaad959d08cf5a42c9', 'request_id': 'req-8051f206-1cb8-4662-bf57-07814172392a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.238500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 880/1799] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:29 2022] DELETE /identity/v3/users/4f40cd02de374369b055214214ff97d6 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:30.239681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:30.241349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:30.366367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585e917-fd48-48a1-ad14-21e8c682f35f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:30.366367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:30.366367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:30.366367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:30.366745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:30.366745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:30.368689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad837b-d12a-4fa6-ba63-275f0a6c1e1c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:30.409527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ebd2a48f9442aaad959d08cf5a42c9', 'request_id': 'req-8051f206-1cb8-4662-bf57-07814172392a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.435714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] The token cache is being invalidate because project 90ebd2a48f9442aaad959d08cf5a42c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:30.458167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.458167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:30.458167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 920/1800] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:29 2022] DELETE /identity/v3/projects/90ebd2a48f9442aaad959d08cf5a42c9 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:30.460643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:30.468426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:30.686152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051f206-1cb8-4662-bf57-07814172392a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:30.709601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:30.711561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:30.712076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:30.712535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] PATH_INFO: `/v3/projects/a04a881f219142a28772322f50e9716b` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:30.714347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04a881f219142a28772322f50e9716b)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:30.736586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.203532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 881/1801] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:30 2022] POST /identity/v3/users => generated 294 bytes in 964 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:31.209662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.209662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.209662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.230132 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:31.231990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.250994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 882/1802] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:31.250994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.250994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.256844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe602d8-10aa-491e-a2d8-2224970104b0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:31.269453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:31.270522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.270807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.271088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.272112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:31.273877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d1e5-f20b-4a91-9520-fb75287307e4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.291936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 883/1803] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:31.293387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.294343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.294769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.314381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04a881f219142a28772322f50e9716b', 'request_id': 'req-da33c315-d3b2-405a-9df8-effefa294786'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:31.325674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:31.327953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04a881f219142a28772322f50e9716b', 'request_id': 'req-da33c315-d3b2-405a-9df8-effefa294786'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:31.328078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.448123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 884/1804] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:31.448123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.466746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:31.467687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04a881f219142a28772322f50e9716b', 'request_id': 'req-da33c315-d3b2-405a-9df8-effefa294786'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:31.487369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665c9d8-9601-4976-9b7d-b3d0a27519f1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:31.491405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] The token cache is being invalidate because project a04a881f219142a28772322f50e9716b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:31.495367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:31.497955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:31.501455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 921/1805] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:30 2022] DELETE /identity/v3/projects/a04a881f219142a28772322f50e9716b => generated 0 bytes in 1042 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:31.513672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.513924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:31.514033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.514111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.514111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.514111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.514539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:31.518112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9471388-0b33-4d53-8bcf-30d5f783bcdb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.543090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 885/1806] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:31.545017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.553829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33c315-d3b2-405a-9df8-effefa294786 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:31.554002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.554002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.556999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:31.557708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.558020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.558335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.567947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:31.568089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:31.569801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665db19-8bb9-4b44-8600-243d3e63ee66 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.574132 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:31.575822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 922/1807] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 923/1808] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:31.612878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.615564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 886/1809] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:31.616754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.628849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.633971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339f014-fa5b-4acf-929f-86fcc8b2ce39 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:31.640665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:31.641712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.642094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.642482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.644451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:31.651089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107cdc1-102f-45ca-a6b8-b031f7431054 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.688322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f456680-c531-4692-87de-d3c9e8213095 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:31.745253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:31.746405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.746619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.746957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] PATH_INFO: `/v3/projects/3a69f4c7362e41beb62ebc790771f846/users/e45e37ea4aa541d098724ad6cd5c9108/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.802801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a69f4c7362e41beb62ebc790771f846, user_id=e45e37ea4aa541d098724ad6cd5c9108, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:31.809240 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aef43d-304f-4561-aaa2-2842c2d03396 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:31.822602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 887/1810] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/projects => generated 393 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:31.824504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.825113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.825523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.866437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:31.867530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:31.975344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 924/1811] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:31 2022] PUT /identity/v3/projects/3a69f4c7362e41beb62ebc790771f846/users/e45e37ea4aa541d098724ad6cd5c9108/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:31.979769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:31.979769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:31.980028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:31.983146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 888/1812] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:31.985498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:31.992788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.016486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183f9c-d950-4af0-8cfb-29f3df217137 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.033101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:32.046766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.201673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.202279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.202815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.205751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] PATH_INFO: `/v3/projects/ee43ff71ee1e4be39168efec8121098f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.212622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee43ff71ee1e4be39168efec8121098f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.214804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.249774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 925/1813] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:32.251787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.288629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8c121-6791-4b1c-821e-274650074e3c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.297010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.298044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.303972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.303972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.303972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.303972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53641a71-8eb8-473d-ae49-735dbbd3ae90 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.384884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee43ff71ee1e4be39168efec8121098f', 'request_id': 'req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:32.399592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee43ff71ee1e4be39168efec8121098f', 'request_id': 'req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:32.413519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee43ff71ee1e4be39168efec8121098f', 'request_id': 'req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:32.482215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 926/1814] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:32.489913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.491646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.492592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.515737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] The token cache is being invalidate because project ee43ff71ee1e4be39168efec8121098f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:32.519045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:32.519045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:32.519045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 889/1815] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:31 2022] DELETE /identity/v3/projects/ee43ff71ee1e4be39168efec8121098f => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:32.519467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.526885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.569951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] MFA Rules not processed for user `e45e37ea4aa541d098724ad6cd5c9108`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:32.576732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.615012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:32.643389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2fc7c-0059-4c77-bdbc-7117a4bdd7d5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.649351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.650227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.655191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.655191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.655191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:32.655191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.655191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0419d82-e110-4a5b-8d7c-f3ada14743d1 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.673868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 890/1816] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:32.673868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.673868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.673868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.717562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] MFA Rules not processed for user `bc4c0fe4d5794e0590060294fcb8964a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:32.719168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.724224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 927/1817] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:32.724224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.724224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.806576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 891/1818] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:32.807689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.809189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.825210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27379-c291-47f1-86d4-b6bc43cded5e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.831587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fb63a-5f8e-4322-949f-a21a90b2d904 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.838556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.841181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.841597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.841990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.844194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.848121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.850281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.851003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.851677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.852969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.854988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.858805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.860717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bef94-6eb4-4774-9d62-116461f7c6a5 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.877000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 928/1819] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:32.879368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.879529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.880064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.907082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 892/1820] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:32.907082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.907482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:32.908713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.923342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 929/1821] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:32.923749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:32.926910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:32.927228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:32.938259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972af7ed-167c-482a-9fa6-07c69f2d951d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.952359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.959354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.959467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64607-01c1-4cb5-bd2b-c758771c4a82 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:32.959564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.959564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.959564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.959564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:32.959867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:32.959867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:32.959867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:32.960102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:32.960707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:32.963079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:33.084336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 893/1822] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:33.085249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:33.086999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:33.106772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 930/1823] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:32 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:33.106772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:33.247724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0909-27fc-401e-8735-a2357cdeae3f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:33.283744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabfe75-d88b-40da-acde-6fe74134fcf3 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:33.296078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:33.298433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.298985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.300094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.301288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:33.303736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dda035-b6b6-421a-9caf-95b917f3b066 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:33.318676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 931/1824] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:33.320997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.321686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.322201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.346732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:33.348216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:33.381784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 932/1825] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:33.381784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:33.400623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff40eec-2d18-43cb-8aef-7d5c76216f16 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:33.407841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:33.409126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.409706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.410350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.411935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:33.416734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:33.442542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:33.443249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.443551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.443814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.444514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:33.446433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:33.907789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 933/1826] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:33 2022] POST /identity/v3/users => generated 284 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:33.907789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:33.908689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 894/1827] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:33 2022] POST /identity/v3/projects => generated 405 bytes in 821 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:33.908689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:33.916429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420ab36-561b-4596-b0c9-b30293a36c13 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:33.991310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0799757-0a84-4c98-9734-7c9e23564fcf admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:33.996735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:33.996905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:33.997890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.997890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.998103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.998189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:33.998354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:33.998625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] PATH_INFO: `/v3/projects/9f158a09f36d4393bcb1bf93b3db8fc7/users/d266513b017446b9b89d382d7a245c24/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:33.998810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.001063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.019152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f158a09f36d4393bcb1bf93b3db8fc7, user_id=d266513b017446b9b89d382d7a245c24, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.021813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.150659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 934/1828] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:33 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:34.151388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.154272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 895/1829] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:33 2022] PUT /identity/v3/projects/9f158a09f36d4393bcb1bf93b3db8fc7/users/d266513b017446b9b89d382d7a245c24/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:34.162823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5c743-14eb-4640-9b8d-b9d66cd48fcc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.163721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.230934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.231637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.231919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.232196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.232927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:34.233236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.234829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.242141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d99c3-b38c-402f-8aea-3b8cdbb028cc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.247454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 935/1830] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:34.247879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.249274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.249407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.249824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.250105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.250925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.253265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.265068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 896/1831] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:34.265205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.265410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de275ca0-c8b5-48af-a0dc-03b48b26abd5 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.272950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.273863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.274245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.274525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.275252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.276792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcea040-46fc-4519-8b86-cd29214f7fe7 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.277284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.281881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.282543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.282823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.283096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.284372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.287534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb3825-8515-4578-9b6f-1096dab6d2df admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.296862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 897/1832] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:34.299234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.299234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.299234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.326311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] MFA Rules not processed for user `d266513b017446b9b89d382d7a245c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:34.327578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:34.391067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 936/1833] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:34 2022] POST /identity/v3/users => generated 309 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:34.391923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.402007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 898/1834] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:34.402868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.404127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2c550-6b8e-49dc-b897-b046eff99872 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.410493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.411182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.411469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.411745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.412560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.414866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9a66-e23c-4a61-8892-59f427d06e45 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.415000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.419992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.420671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.420960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.421240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.421978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.423697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.442592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 899/1835] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:34.442592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.448690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7f2d-068c-413b-b159-84e3504bffdb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.453991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.454910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.455224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.455502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] PATH_INFO: `/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f/users/e634d5725f31429a8282a61cb8f77c1f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.474754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b2bd982ba543e6ac0f9905abd4459f, user_id=e634d5725f31429a8282a61cb8f77c1f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.493175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.551852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 937/1836] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:34 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:34.551852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.587708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 900/1837] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:34 2022] PUT /identity/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f/users/e634d5725f31429a8282a61cb8f77c1f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:34.588609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.633709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b24cd-5200-4f85-a6de-ad66efc4d918 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.704417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd1d2-5e1e-4966-8e68-e43003a61cc3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.710245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:34.729397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.729397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.729397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.729397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.729765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.729765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.763146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.764408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.764697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.765398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.766169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.769107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.791485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 938/1838] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:34.792836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.804470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0d966-712f-4d28-8f1b-c90c56506992 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.810997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.812646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.812944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.813228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] PATH_INFO: `/v3/projects/9248c9f961ef46b6bb0a3084bdd36e0a/users/09524b5e841f44588f9d14aa3cbee639/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.833864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9248c9f961ef46b6bb0a3084bdd36e0a, user_id=09524b5e841f44588f9d14aa3cbee639, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.837243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.847044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 901/1839] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:34 2022] POST /identity/v3/projects => generated 393 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:34.847044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:34.914688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43276ad-499b-4012-baee-a1d6b4354b6c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:34.919618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:34.920353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:34.920661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:34.920986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:34.921737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:34.924091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:34.977847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 939/1840] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:34 2022] PUT /identity/v3/projects/9248c9f961ef46b6bb0a3084bdd36e0a/users/09524b5e841f44588f9d14aa3cbee639/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:34.978799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.042529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed88569-e110-4bee-b1b2-f04db805c14c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.047296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.047967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.048254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.048534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.049265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.051619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.061651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 940/1841] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:35.062539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.073005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272587-e190-4edf-b5b2-3e073e965ee6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.077828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.078788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.079217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.079566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] PATH_INFO: `/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8/users/d831d2d91c484bddac66647dc252dfd7/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.098414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] RBAC: Authorizing `identity:create_grant(project_id=94781d8d5d4b4bf19ede96f7a5be86a8, user_id=d831d2d91c484bddac66647dc252dfd7, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.101511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef9723-3674-4bfd-ba97-905cbde9feff admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.128214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 902/1842] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:34 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:35.128214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.141933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e48-8624-456d-8129-630a6fbdc8e5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.149701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.150488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.150837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.151238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.151924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.153854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.242717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 941/1843] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:35 2022] PUT /identity/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8/users/d831d2d91c484bddac66647dc252dfd7/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:35.244065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.244357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.244631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.274184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] MFA Rules not processed for user `09524b5e841f44588f9d14aa3cbee639`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:35.275439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:35.336411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 903/1844] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:35 2022] POST /identity/v3/users => generated 303 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:35.336411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.344156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 942/1845] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:35.344551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b214-9cd4-45cf-ae12-fad8a5c28410 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.344998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.404952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.405787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.406077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.406359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] PATH_INFO: `/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f/users/e634d5725f31429a8282a61cb8f77c1f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.408404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b2bd982ba543e6ac0f9905abd4459f, user_id=e634d5725f31429a8282a61cb8f77c1f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.410140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e08a-290a-4abf-8dfd-10c3f68bcaf8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.411142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.414983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.415685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.415965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.416238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.417009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.418622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.421987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 904/1846] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:35 2022] PUT /identity/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f/users/e634d5725f31429a8282a61cb8f77c1f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:35.423066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.433048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 943/1847] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:35.433048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.441728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2fc6-ea73-4a3f-b8dd-6764e1332657 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.500307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a465b2-d171-4f71-a5e6-2489979f2e89 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.500925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.501607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.502111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.503175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.504120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.505911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:35.506596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.507293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.507579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.507859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.507958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.508587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.510243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab6239-114c-402e-a33f-7c0739445e5d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.518086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 944/1848] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:35.519028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.521736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 905/1849] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:35.523294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.523665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.523967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.529534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fe793-ff9f-4044-9389-38c61c54589a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.534233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.534887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.535195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.535490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.536243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.538147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.552966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] MFA Rules not processed for user `e634d5725f31429a8282a61cb8f77c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:35.554291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:35.627333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 906/1850] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:35.628917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.642966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db38629-59e8-44cf-9043-c2d548d84eb2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.648627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.649397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.649584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.649944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.650810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.652346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.666725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 907/1851] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:35.666725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.674936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b228f-93f0-4141-904c-cfde44fb23f2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.680940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.681849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.682129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.682403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] PATH_INFO: `/v3/projects/2327af55bb0a4ad2ae8817a266b75425/users/2699763c94b34212a73788b41c039340/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.711846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2327af55bb0a4ad2ae8817a266b75425, user_id=2699763c94b34212a73788b41c039340, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.711963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 945/1852] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:35 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:35.711963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.712129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.778527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f792-75e8-4299-8e6c-698587cac821 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.787002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.787002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.787002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.787002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] PATH_INFO: `/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8/users/d831d2d91c484bddac66647dc252dfd7/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.792304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94781d8d5d4b4bf19ede96f7a5be86a8, user_id=d831d2d91c484bddac66647dc252dfd7, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.795074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.807203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 946/1853] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:35 2022] PUT /identity/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8/users/d831d2d91c484bddac66647dc252dfd7/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:35.808144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.809878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:35.869123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67568c-3889-4cf9-86c2-5a7ba39cc5e7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.872675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.873316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.873639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.873917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.882603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 908/1854] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:35 2022] PUT /identity/v3/projects/2327af55bb0a4ad2ae8817a266b75425/users/2699763c94b34212a73788b41c039340/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:35.884565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.950120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.951892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:35.963572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 947/1855] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:35.964890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:35.965811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f4810-0677-4b0b-a684-85b5a2b3f486 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:35.971397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:35.973831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:35.973831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:35.973831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] PATH_INFO: `/v3/projects/67aaaf0063044155a845e44eea2dcad4/users/874042de57cb4e919f995ecb287d1ff2/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:35.992619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67aaaf0063044155a845e44eea2dcad4, user_id=874042de57cb4e919f995ecb287d1ff2, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:35.995885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.055581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e40ae-4de8-47b1-9515-a48e93c2d0b0 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.062632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.063459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.063849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.064140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.067761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.070001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.131871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 909/1856] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:35 2022] PUT /identity/v3/projects/67aaaf0063044155a845e44eea2dcad4/users/874042de57cb4e919f995ecb287d1ff2/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:36.142172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.195488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 948/1857] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:35 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:36.195488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.202981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3400f-fd3b-4940-8708-8d1b940a5496 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.218493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d363-9a09-4267-a173-10286bbb74f6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.226293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.227359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.234335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.234335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.234335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.234335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394556c-aaae-4b7c-89b1-fbd66b503892 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.242101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 910/1858] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:36.246370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.246749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.247091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.277330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] MFA Rules not processed for user `874042de57cb4e919f995ecb287d1ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:36.278753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:36.305653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.307714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.309491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.309491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] PATH_INFO: `/v3/users/e45e37ea4aa541d098724ad6cd5c9108` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.312004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45e37ea4aa541d098724ad6cd5c9108)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.315942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.374083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 911/1859] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:36.374956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.386661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dd9d4-3365-4cac-8b15-f5ff5afb7329 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.391286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.391959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.392247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.392526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.393981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.395679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.406530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 912/1860] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:36.407478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.418200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d259d-02f2-43f3-a8f2-2a6fdfb05e78 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.422882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.423810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.424097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.424909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] PATH_INFO: `/v3/projects/2327af55bb0a4ad2ae8817a266b75425/users/2699763c94b34212a73788b41c039340/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.426967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2327af55bb0a4ad2ae8817a266b75425, user_id=2699763c94b34212a73788b41c039340, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.429673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.441337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 913/1861] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:36 2022] PUT /identity/v3/projects/2327af55bb0a4ad2ae8817a266b75425/users/2699763c94b34212a73788b41c039340/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:36.442815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.510287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9539c1-139a-49e2-8b04-ba08bd432395 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.515462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:36.516154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.516442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.516719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.517420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:36.519513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:36.768068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45e37ea4aa541d098724ad6cd5c9108'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:36.776690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45e37ea4aa541d098724ad6cd5c9108'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:36.795875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45e37ea4aa541d098724ad6cd5c9108'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:36.796662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45e37ea4aa541d098724ad6cd5c9108'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:36.861091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 914/1862] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:36 2022] POST /identity/v3/projects => generated 385 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:36.861091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:36.863065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:36.910266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-327362c1-12ca-4452-8060-b84b3b144dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45e37ea4aa541d098724ad6cd5c9108'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:36.920539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 949/1863] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:36 2022] DELETE /identity/v3/users/e45e37ea4aa541d098724ad6cd5c9108 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:36.922107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:36.922396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:36.922804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:36.969146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] MFA Rules not processed for user `2699763c94b34212a73788b41c039340`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:36.969313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46bcda-c79c-447a-8b1a-b848f9aab4b0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:36.970001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.087262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.088213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.088505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.088786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] PATH_INFO: `/v3/projects/7c254132b4d847d4a42292b731356447/users/cac587d6bcae46f7bc4d086fd6a5c70b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.100643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 950/1864] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:37.103612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.107222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.128496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dca8a0-9fa7-45ed-98b4-538b7c0e7f68 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.139721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c254132b4d847d4a42292b731356447, user_id=cac587d6bcae46f7bc4d086fd6a5c70b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.140682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.144891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.145181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.145460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.146166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.148679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.149493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961a57e-99cd-4c99-a9c8-bf131a2dc0a0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.283198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 951/1865] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:37 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:37.283198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.283198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.283198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.286276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 915/1866] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:36 2022] PUT /identity/v3/projects/7c254132b4d847d4a42292b731356447/users/cac587d6bcae46f7bc4d086fd6a5c70b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:37.287625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.290126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.319002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:37.319002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.375137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c8cc0-5a9c-4444-9df0-b33440d6a65a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.379923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:37.385025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.398920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.398920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.398920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.404911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 952/1867] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:37.406031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.422264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:37.431301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3ae41-7432-490b-8dab-9c0ae39e890e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.436714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.437394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.437689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.437972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.438716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.440938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.454383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 953/1868] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:37.457666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.469493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda5346-2354-4f05-abb1-4d736c2c4960 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.475994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.476879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.477495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.477692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.478043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.478468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.479233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.481327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.493607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 954/1869] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:37.494620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.495248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 916/1870] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:37.496310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.501626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.509130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ab53-b2c4-4d54-a56a-4b2d1d78c777 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.516814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.517525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.517811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.518718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.520260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.521923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.534382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 917/1871] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:37.539718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.554220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03089-afc3-44e6-ae8b-22b96e7443a9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.559579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.560673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.562150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.562533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] PATH_INFO: `/v3/projects/7c254132b4d847d4a42292b731356447/users/cac587d6bcae46f7bc4d086fd6a5c70b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.565432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c254132b4d847d4a42292b731356447, user_id=cac587d6bcae46f7bc4d086fd6a5c70b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.568778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.582859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 918/1872] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:37 2022] PUT /identity/v3/projects/7c254132b4d847d4a42292b731356447/users/cac587d6bcae46f7bc4d086fd6a5c70b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:37.584979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1812658-6ba3-4a9e-900f-7e24bb38b92d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.588023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.644427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.645093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.645378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.645664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.673306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10499e2-3dff-46c9-8b1a-cea4a8df72a4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.681478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.683883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.683883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.683883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] PATH_INFO: `/v3/projects/40d1f3d2afd646eb88d645cde8b6dc30/users/e561baf4add24569a23922f81b6fbcf1/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.701354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d1f3d2afd646eb88d645cde8b6dc30, user_id=e561baf4add24569a23922f81b6fbcf1, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.704559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b58b28-3448-4f86-9c6c-0516c66c333c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.723073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.725473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.735888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 955/1873] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:37.737556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.738966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.819989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 919/1874] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:37 2022] PUT /identity/v3/projects/40d1f3d2afd646eb88d645cde8b6dc30/users/e561baf4add24569a23922f81b6fbcf1/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:37.819989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b4405-0cee-4c87-b0e5-0b21fb65c774 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.819989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b4405-0cee-4c87-b0e5-0b21fb65c774 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.819989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b4405-0cee-4c87-b0e5-0b21fb65c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.820394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ad6f5-afd6-4a76-aebf-48c384391018 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.854698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5a1b4405-0cee-4c87-b0e5-0b21fb65c774 None None] MFA Rules not processed for user `cac587d6bcae46f7bc4d086fd6a5c70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:37.856169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5a1b4405-0cee-4c87-b0e5-0b21fb65c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.890832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.892304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.892602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.893051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.895586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.931527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 920/1875] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:37.932839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.933125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.933400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.969185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] MFA Rules not processed for user `e561baf4add24569a23922f81b6fbcf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:37.971065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.977703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.979959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.991439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 956/1876] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:37.992289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.001850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf52-7fe3-4d5c-8622-e7a3995fda08 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.007665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.009227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.009576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.009946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.021370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.023159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.051145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 957/1877] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:38.081462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 921/1878] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:38.086058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.170971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44e0ec-2782-4cbd-a3e5-31382779efd5 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.187298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.187987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.188272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.188549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.189399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.195106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.394233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.399237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:38.417300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 958/1879] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:38 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:38.428500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.436476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5beda4-bffe-4f26-9042-3227d9dc872f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.522081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.522948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.523275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.523771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] PATH_INFO: `/v3/projects/3a69f4c7362e41beb62ebc790771f846` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.528658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a69f4c7362e41beb62ebc790771f846)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.531355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.536456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dc19e-b303-47f0-ac96-20ffef28f041 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.542607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.543944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.544117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.544483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.545431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.548079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89694-a1d7-4177-8b34-e7d554993078 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.667676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 959/1880] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:38 2022] POST /identity/v3/users => generated 299 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:38.670005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.670438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.670727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.700611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] MFA Rules not processed for user `d831d2d91c484bddac66647dc252dfd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:38.701972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:38.727673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a69f4c7362e41beb62ebc790771f846', 'request_id': 'req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:38.733290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a69f4c7362e41beb62ebc790771f846', 'request_id': 'req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:38.741680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a69f4c7362e41beb62ebc790771f846', 'request_id': 'req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:38.785359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 960/1881] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:38.786230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.799725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad9ff4-ea34-4e19-a63b-235bcafdf3dc None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.853431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] The token cache is being invalidate because project 3a69f4c7362e41beb62ebc790771f846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:38.856002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:38.857629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:38.871735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 922/1882] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:38 2022] DELETE /identity/v3/projects/3a69f4c7362e41beb62ebc790771f846 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:38.872644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.875471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:38.896082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.897106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.901415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.901415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.901415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.901415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.916311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 961/1883] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:38.920206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:38.927070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:38.953244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75449712-dc3d-48e6-9e3e-dca93495c3cd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:38.960948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:38.962142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:38.962541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:38.962864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] PATH_INFO: `/v3/projects/147905762c6d4c789b3a68d82b0a5cc7/users/71039acecc964457b3b3034dda7c9a65/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:38.979827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] RBAC: Authorizing `identity:create_grant(project_id=147905762c6d4c789b3a68d82b0a5cc7, user_id=71039acecc964457b3b3034dda7c9a65, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:38.982757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:38.997949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b0a1-0cb7-4f6f-af71-01e0bfb2b47d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:39.003498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:39.004570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.004859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.005202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] PATH_INFO: `/v3/users/e634d5725f31429a8282a61cb8f77c1f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.006487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e634d5725f31429a8282a61cb8f77c1f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:39.008465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:39.066262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 962/1884] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:38 2022] PUT /identity/v3/projects/147905762c6d4c789b3a68d82b0a5cc7/users/71039acecc964457b3b3034dda7c9a65/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:39.070455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:39.149398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9394-124b-451c-8b4e-0614afc6876f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:39.159458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:39.159932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.160220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.160527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.161244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:39.163393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:39.178710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 963/1885] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:39.179666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:39.202720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07487-37dc-4990-bf0f-72e4fc0ec0f8 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:39.210537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:39.211516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.211835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.212117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] PATH_INFO: `/v3/projects/147905762c6d4c789b3a68d82b0a5cc7/users/71039acecc964457b3b3034dda7c9a65/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.214439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147905762c6d4c789b3a68d82b0a5cc7, user_id=71039acecc964457b3b3034dda7c9a65, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:39.217424 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c049b1-b809-4724-b7cb-a09d7739d9f8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:39.235213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 964/1886] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:39 2022] PUT /identity/v3/projects/147905762c6d4c789b3a68d82b0a5cc7/users/71039acecc964457b3b3034dda7c9a65/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:39.241690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.242303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.242929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.279905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] MFA Rules not processed for user `71039acecc964457b3b3034dda7c9a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:39.281907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:39.289254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e634d5725f31429a8282a61cb8f77c1f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:39.290424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e634d5725f31429a8282a61cb8f77c1f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:39.313880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e634d5725f31429a8282a61cb8f77c1f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:39.326635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e634d5725f31429a8282a61cb8f77c1f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:39.381286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 965/1887] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:39.394540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:39.401736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:39.418373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-769adddc-b132-4a41-8b02-b171d639335a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e634d5725f31429a8282a61cb8f77c1f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:39.426523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 923/1888] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:38 2022] DELETE /identity/v3/users/e634d5725f31429a8282a61cb8f77c1f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:39.474249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aba19-03af-4eca-9997-195122643ea4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:39.542844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:39.543904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.544303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.544589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.628446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:39.629905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f4eb9-3999-459e-ad34-cfe69b21b4b5 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:39.640283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 966/1889] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:39.710665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.711111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.711654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.713215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 924/1890] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:39.726754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.727064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.727384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.754762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:39.756049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:39.875257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 967/1891] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:39.887784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:39.896399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:39.917024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd18c9-588f-4461-bf34-dc8d878c4c3c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:39.922396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:39.923027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:39.923355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:39.923675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:39.944533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:39.946182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afe73-ea45-4562-bef0-bf50d7cc9fbd service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:39.958464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 925/1892] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:39.995013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:39.996929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:40.123645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4701ea-6ba8-42e5-ac8b-5b903e7857d4 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:40.130995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:40.132641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:40.132923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:40.133480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] PATH_INFO: `/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:40.135286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b2bd982ba543e6ac0f9905abd4459f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:40.137735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:40.274466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:40.292008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b2bd982ba543e6ac0f9905abd4459f', 'request_id': 'req-c465694c-44ef-4fd7-bc93-db5f8fb610b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:40.297511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b2bd982ba543e6ac0f9905abd4459f', 'request_id': 'req-c465694c-44ef-4fd7-bc93-db5f8fb610b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:40.401675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b2bd982ba543e6ac0f9905abd4459f', 'request_id': 'req-c465694c-44ef-4fd7-bc93-db5f8fb610b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:40.417892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] The token cache is being invalidate because project c6b2bd982ba543e6ac0f9905abd4459f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:40.418304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:40.420686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c465694c-44ef-4fd7-bc93-db5f8fb610b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:40.433165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:40.433675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 968/1893] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:39 2022] DELETE /identity/v3/projects/c6b2bd982ba543e6ac0f9905abd4459f => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:40.434326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:40.435377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:40.462174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:40.463714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:40.544011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 926/1894] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:40.555715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:40.556166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:40.556448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:40.581547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:40.582941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:40.593316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 969/1895] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:40.597758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:40.609930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28df5c-fdf8-4486-838e-29b4b047cdfc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:40.624727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:40.625559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:40.625852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:40.626158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:40.627703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:40.629346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:40.630548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:40.647009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 927/1896] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:17:40.652537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:40.671834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93bdc1-d1c7-40c4-a4ac-0d8eb83684fd None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:40.678395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:40.679184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:40.679633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:40.680047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:40.682180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:40.685556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86741e-b077-46a7-8cc7-e0a04131fde7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:40.847876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 970/1897] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:40 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:40.852108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:40.866921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda47057-7b76-425c-b8e8-16a433df334f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.051083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.052801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.053101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.054946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.055792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.056700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff39a3-1082-4d03-9d38-26bf11af2b53 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.057096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff39a3-1082-4d03-9d38-26bf11af2b53 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.057477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff39a3-1082-4d03-9d38-26bf11af2b53 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.059213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 971/1898] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:17:41.059832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.076050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.076341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.076612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.102720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] MFA Rules not processed for user `aaf80b4722094475934134be619652e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:41.103989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.166431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:41.214704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 972/1899] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:41.235804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 928/1900] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:40 2022] POST /identity/v3/users => generated 295 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:41.235907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.238336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.239024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.258127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd145f-29eb-4178-b70a-c94898828f2e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.265656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.266658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.266904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.267317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.269055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.269171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bdd1a-107a-40f6-a331-acd07694d2af None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.271186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.285059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 929/1901] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:41.286331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.291401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.306591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.310259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.311215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.311647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.325889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.328200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.343864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 973/1902] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:41.355325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.361473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895546f9-a494-4cac-95bd-43c977744486 service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.368284 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.368972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.369264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.369547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.370292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.374606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.391404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 974/1903] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:41.393150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.406553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.416812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11258cc5-f761-4a83-801f-b8b6178980c6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.426697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.427605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.437224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.437224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.437877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1648-258e-4818-8735-0e8fccd001af admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.437960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.444061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.445126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.445552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.446497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] PATH_INFO: `/v3/users/bc4c0fe4d5794e0590060294fcb8964a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.448914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4c0fe4d5794e0590060294fcb8964a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.452823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.548211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.551440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.562817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 930/1904] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:41.565412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:41.608316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cf870-c1c7-48b5-ac22-74ecbb5e98cd service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:41.620580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:41.621734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.622190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.622604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] PATH_INFO: `/v3/projects/c337fe169abf42ed9c8dd68157de3830/users/c810e88dcd144bcc94488690b1a8ffe1/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.656967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c337fe169abf42ed9c8dd68157de3830, user_id=c810e88dcd144bcc94488690b1a8ffe1, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:41.661935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaeeee-b81a-4d82-a870-60b177ec662f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:41.784373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 931/1905] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:41 2022] PUT /identity/v3/projects/c337fe169abf42ed9c8dd68157de3830/users/c810e88dcd144bcc94488690b1a8ffe1/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:41.795590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:41.795888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:41.809987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:41.811163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4c0fe4d5794e0590060294fcb8964a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:41.823764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4c0fe4d5794e0590060294fcb8964a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:41.852288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4c0fe4d5794e0590060294fcb8964a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:41.854003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4c0fe4d5794e0590060294fcb8964a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:41.902906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] MFA Rules not processed for user `c810e88dcd144bcc94488690b1a8ffe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:41.905476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:41.920437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4c0fe4d5794e0590060294fcb8964a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:41.929191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 975/1906] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:41 2022] DELETE /identity/v3/users/bc4c0fe4d5794e0590060294fcb8964a => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:42.024135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:42.110725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 932/1907] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:42.115704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:42.119340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:42.242678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddf6b8-28cd-4396-8a2f-dadb7e51a8f4 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:42.247751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:42.248610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:42.249029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:42.249468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:42.250323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:42.253056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:42.376386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 976/1908] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:42 2022] POST /identity/v3/projects => generated 387 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:42.381147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:42.398111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:42.400048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:42.558068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3dc43-9805-439b-afd8-e6da9d07ffe0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:42.578950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:42.579694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:42.580540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:42.583164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:42.585580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:42.588086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951047ad-86ef-4523-9816-d46cef86d764 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:42.589679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:42.617761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:42.621130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:42.621130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:42.621130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] PATH_INFO: `/v3/projects/586d9118a6c647ff91f9ab80efaee9a3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:42.622665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=586d9118a6c647ff91f9ab80efaee9a3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:42.624816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:42.743158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 933/1909] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:42 2022] POST /identity/v3/users => generated 300 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:42.744114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:42.780253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff763af-1172-4ea2-bd2e-1b0c09fa0bd9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:42.791238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:42.795377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:42.795819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:42.797812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:42.800359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:42.804654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:42.809513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586d9118a6c647ff91f9ab80efaee9a3', 'request_id': 'req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:42.825799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586d9118a6c647ff91f9ab80efaee9a3', 'request_id': 'req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:42.829271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 934/1910] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:42.836842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:42.849158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62147c-9591-4ef7-9d92-478bec4b5392 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:42.920440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:42.921613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:42.921895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:42.922325 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] PATH_INFO: `/v3/projects/49c1f6b87632493db7de3f7cceba77ba/users/7ea1b3c6937a49cca4b1ff398bd9cf5b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:42.939371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c1f6b87632493db7de3f7cceba77ba, user_id=7ea1b3c6937a49cca4b1ff398bd9cf5b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:42.942973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:42.994934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586d9118a6c647ff91f9ab80efaee9a3', 'request_id': 'req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.013587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] The token cache is being invalidate because project 586d9118a6c647ff91f9ab80efaee9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:43.013587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.013888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.015634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 977/1911] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:42 2022] DELETE /identity/v3/projects/586d9118a6c647ff91f9ab80efaee9a3 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:43.022071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.023771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.042829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c2e2-fbde-4274-a60b-f6678ac3ccf0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:43.053209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 935/1912] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:42 2022] PUT /identity/v3/projects/49c1f6b87632493db7de3f7cceba77ba/users/7ea1b3c6937a49cca4b1ff398bd9cf5b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:43.063310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.066468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.140864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:43.141753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.142092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.142413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] PATH_INFO: `/v3/users/d831d2d91c484bddac66647dc252dfd7` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.143910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d831d2d91c484bddac66647dc252dfd7)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:43.145868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:43.147707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79762f2-35d2-4521-8a72-1d56627aaa03 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:43.153459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:43.154220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.154577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.154924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.155768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:43.157473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:43.168894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 936/1913] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:43.173164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.184337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced3d19-907b-4024-8a24-1b7597d36deb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:43.190225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:43.191286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.191574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.191855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] PATH_INFO: `/v3/projects/49c1f6b87632493db7de3f7cceba77ba/users/7ea1b3c6937a49cca4b1ff398bd9cf5b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.194312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c1f6b87632493db7de3f7cceba77ba, user_id=7ea1b3c6937a49cca4b1ff398bd9cf5b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:43.197170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9b7f-4a53-4233-8878-4b8166f5cfb8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:43.210580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 937/1914] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:43 2022] PUT /identity/v3/projects/49c1f6b87632493db7de3f7cceba77ba/users/7ea1b3c6937a49cca4b1ff398bd9cf5b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:43.222203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.222661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.227174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.268799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] MFA Rules not processed for user `7ea1b3c6937a49cca4b1ff398bd9cf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:43.270993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.351302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 938/1915] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:43.387253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd831d2d91c484bddac66647dc252dfd7'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.396642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd831d2d91c484bddac66647dc252dfd7'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.410158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.411795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.413120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd831d2d91c484bddac66647dc252dfd7'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.413969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd831d2d91c484bddac66647dc252dfd7'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.475979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ef961-72e9-4be2-8663-4bec7c32d45f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:43.482859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:43.485073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.485073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.485073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.487027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.497807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd831d2d91c484bddac66647dc252dfd7'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:43.500181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:43.502415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:43.505968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 978/1916] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:43 2022] DELETE /identity/v3/users/d831d2d91c484bddac66647dc252dfd7 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:43.524918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 939/1917] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:43.543952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=84,keystone:INSERT=5,keystone:DELETE=11 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:43.675037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:43.802242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.803897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.890718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8cf-6f5f-4d03-8507-6403497e9fed admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:43.894804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:43.898890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:43.915028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:43.915969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:43.916251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:43.916527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] PATH_INFO: `/v3/users/09524b5e841f44588f9d14aa3cbee639` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:43.920146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09524b5e841f44588f9d14aa3cbee639)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:43.925962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:43.998146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1df8de-3222-4c93-9e18-f3b01726dbdf service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.003838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:44.004776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.005066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.005391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] PATH_INFO: `/v3/users/c810e88dcd144bcc94488690b1a8ffe1` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.006928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c810e88dcd144bcc94488690b1a8ffe1)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:44.009249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:44.264042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c810e88dcd144bcc94488690b1a8ffe1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.264728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c810e88dcd144bcc94488690b1a8ffe1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.266587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09524b5e841f44588f9d14aa3cbee639'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.278747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09524b5e841f44588f9d14aa3cbee639'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.279081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c810e88dcd144bcc94488690b1a8ffe1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.292902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c810e88dcd144bcc94488690b1a8ffe1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.296731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09524b5e841f44588f9d14aa3cbee639'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.297482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09524b5e841f44588f9d14aa3cbee639'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.411696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c810e88dcd144bcc94488690b1a8ffe1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.418258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 940/1918] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:43 2022] DELETE /identity/v3/users/c810e88dcd144bcc94488690b1a8ffe1 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:44.419066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:44.420449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.486995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdbac5-8930-40c5-8033-469dc9178e68 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.492094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:44.492975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.493264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.493547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] PATH_INFO: `/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.494773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94781d8d5d4b4bf19ede96f7a5be86a8)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:44.496741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:44.514813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-04103670-4fae-4886-8ee6-fc43d9389a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09524b5e841f44588f9d14aa3cbee639'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.524228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 979/1919] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:43 2022] DELETE /identity/v3/users/09524b5e841f44588f9d14aa3cbee639 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:44.529172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eb1b8-f072-4a05-a9c1-15c7b0a23963 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.529544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eb1b8-f072-4a05-a9c1-15c7b0a23963 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.529900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eb1b8-f072-4a05-a9c1-15c7b0a23963 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.531648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 980/1920] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:17:44.532998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cafd6-8cbc-401d-8fc8-552449c132a9 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.533355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cafd6-8cbc-401d-8fc8-552449c132a9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.533727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cafd6-8cbc-401d-8fc8-552449c132a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.557639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bd4cafd6-8cbc-401d-8fc8-552449c132a9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:44.558546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bd4cafd6-8cbc-401d-8fc8-552449c132a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.620993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 981/1921] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:44.622380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a51af-e32d-4168-ba55-c1cda39c19e0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.622674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a51af-e32d-4168-ba55-c1cda39c19e0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.622985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a51af-e32d-4168-ba55-c1cda39c19e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.642013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-854a51af-e32d-4168-ba55-c1cda39c19e0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:44.643799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94781d8d5d4b4bf19ede96f7a5be86a8', 'request_id': 'req-675f8258-d649-45bd-9d33-78277ddd5fa6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.644620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-854a51af-e32d-4168-ba55-c1cda39c19e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.649241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94781d8d5d4b4bf19ede96f7a5be86a8', 'request_id': 'req-675f8258-d649-45bd-9d33-78277ddd5fa6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.656540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94781d8d5d4b4bf19ede96f7a5be86a8', 'request_id': 'req-675f8258-d649-45bd-9d33-78277ddd5fa6'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.705036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 982/1922] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:44.706507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.706690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.706973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.731190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:44.733436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.748736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] The token cache is being invalidate because project 94781d8d5d4b4bf19ede96f7a5be86a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:44.749259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.750206 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:44.752597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 941/1923] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:44 2022] DELETE /identity/v3/projects/94781d8d5d4b4bf19ede96f7a5be86a8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:44.753667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:44.757920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.826911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f8258-d649-45bd-9d33-78277ddd5fa6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.832724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:44.833376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.833776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.834119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.848461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.848974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 983/1924] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:44.848974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:44.848974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.852456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:44.854215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753dd069-b054-494e-8e49-70f2fc5037ca service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:44.861749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948f910-445a-44fa-9ef5-cf5dd3f929ea None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.866587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 942/1925] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:44.866970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:44.867770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.867984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42288348-7703-4d7a-a740-86ca3be8a8ee None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.868284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42288348-7703-4d7a-a740-86ca3be8a8ee None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.868669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.868778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42288348-7703-4d7a-a740-86ca3be8a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.869139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.876413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.888185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:44.889764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:44.890032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-42288348-7703-4d7a-a740-86ca3be8a8ee None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:44.891365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-42288348-7703-4d7a-a740-86ca3be8a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.903380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 984/1926] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:44.905541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:44.907173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.911019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 943/1927] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:44.918369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.919321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.919614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.951174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67aecdc-1ccd-4b92-afda-5247fb0e01b8 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.960924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:44.961734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:44.962175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:44.962439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:44.967452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:44.968891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:44.975204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:44.977445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:44.979460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 944/1928] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:44.983285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:44.994930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34269c-ccd0-4b7f-b83e-b120bb106f94 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:44.996257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 985/1929] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:45.009378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.009979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.011674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.011984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.014691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:45.015036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.016743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.045198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.057517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 945/1930] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:45.061189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646815f2-f356-4ee0-a0eb-8e414ddda21b service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.073645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.075139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.075620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.076103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.076978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.079430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.229218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 986/1931] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:45 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:45.233413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.246149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1ff71-da8f-4952-97ce-c55445219762 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.339456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.346797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:45.357075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.357075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.357075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.357075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.357075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.467374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 946/1932] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:45 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:45.468579 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.480775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05784ca7-5713-4dc2-b7e1-e96078181081 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.491574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.491574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.491574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.491574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.492066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.492143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.505935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 987/1933] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:45.509670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.522568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc266de-a6e6-4ad1-9e63-7be9fafa6199 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.529706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.530408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.530754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.530974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.531873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.535495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.558478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 947/1934] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:45.562345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.578837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68861fbf-2c33-4553-a8c5-4d69681591d4 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.583104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.584696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.585305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.585838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] PATH_INFO: `/v3/projects/25d02814ccef4b4cb90db19c0eb21ee4/users/4a4323ceb232418ea76f01545b92571b/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.608148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d02814ccef4b4cb90db19c0eb21ee4, user_id=4a4323ceb232418ea76f01545b92571b, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.611836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f04d3-1a7c-4f1c-803e-f756c1790021 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.670081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.671935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:45.686348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e58d2-42ad-4d7a-a072-94309603faec admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.691869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.693040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.693040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.693460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] PATH_INFO: `/v3/users/d266513b017446b9b89d382d7a245c24` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.694744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d266513b017446b9b89d382d7a245c24)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.696866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.721100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 988/1935] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:45 2022] PUT /identity/v3/projects/25d02814ccef4b4cb90db19c0eb21ee4/users/4a4323ceb232418ea76f01545b92571b/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:45.732794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.733087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.733592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.766514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] MFA Rules not processed for user `4a4323ceb232418ea76f01545b92571b`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:45.768558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:45.836586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 989/1936] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:45.837460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:45.838723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:45.912894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d95f8-2498-44f5-94e1-380dd9d2eacb None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:45.917599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:45.918560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:45.918847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:45.919471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] PATH_INFO: `/v3/projects/9248c9f961ef46b6bb0a3084bdd36e0a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:45.920855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9248c9f961ef46b6bb0a3084bdd36e0a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:45.922814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:45.966854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd266513b017446b9b89d382d7a245c24'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:45.967565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd266513b017446b9b89d382d7a245c24'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:45.979781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd266513b017446b9b89d382d7a245c24'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:45.991649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd266513b017446b9b89d382d7a245c24'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.103684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd266513b017446b9b89d382d7a245c24'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.110493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 948/1937] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:45 2022] DELETE /identity/v3/users/d266513b017446b9b89d382d7a245c24 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:46.112445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:46.114040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.121888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:46.127985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9248c9f961ef46b6bb0a3084bdd36e0a', 'request_id': 'req-353b80cb-c855-4ee8-9257-944f551843e6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.134307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9248c9f961ef46b6bb0a3084bdd36e0a', 'request_id': 'req-353b80cb-c855-4ee8-9257-944f551843e6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.185387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27832d07-c1cf-41cd-b3ac-431301c51148 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:46.191103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:46.191801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.192087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.192366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.193111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:46.195038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361b61-42fa-43ba-99c8-015063c269e8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:46.223688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9248c9f961ef46b6bb0a3084bdd36e0a', 'request_id': 'req-353b80cb-c855-4ee8-9257-944f551843e6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.242915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] The token cache is being invalidate because project 9248c9f961ef46b6bb0a3084bdd36e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:46.243263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.244041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.245938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 990/1938] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:45 2022] DELETE /identity/v3/projects/9248c9f961ef46b6bb0a3084bdd36e0a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:46.246658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:46.248058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.299457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 949/1939] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:46 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:46.301039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.301413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.301778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.329274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:46.331705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.334244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b80cb-c855-4ee8-9257-944f551843e6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:46.403194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 950/1940] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:46.404038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:46.405370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.411322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:46.412837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.413318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.413598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] PATH_INFO: `/v3/projects/c337fe169abf42ed9c8dd68157de3830` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.414856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c337fe169abf42ed9c8dd68157de3830)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:46.417116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:46.418799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad288-8b11-40f9-9ca2-1d20ac787631 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:46.424009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:46.425180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.425527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.425874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] PATH_INFO: `/v3/users/cac587d6bcae46f7bc4d086fd6a5c70b` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.427343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac587d6bcae46f7bc4d086fd6a5c70b)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:46.429740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:46.576754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c337fe169abf42ed9c8dd68157de3830', 'request_id': 'req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.584381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c337fe169abf42ed9c8dd68157de3830', 'request_id': 'req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.709271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac587d6bcae46f7bc4d086fd6a5c70b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.709956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac587d6bcae46f7bc4d086fd6a5c70b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.725254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac587d6bcae46f7bc4d086fd6a5c70b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.733530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac587d6bcae46f7bc4d086fd6a5c70b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.788984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c337fe169abf42ed9c8dd68157de3830', 'request_id': 'req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.803440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] The token cache is being invalidate because project c337fe169abf42ed9c8dd68157de3830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:46.803843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.804567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.806489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 991/1941] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:46 2022] DELETE /identity/v3/projects/c337fe169abf42ed9c8dd68157de3830 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:46.807034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:46.812348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.884606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf6fff-e08c-41fe-a43e-c5ec553a08cf admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:46.885864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac587d6bcae46f7bc4d086fd6a5c70b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:46.891069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:46.891949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.892294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.895184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.895184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:46.895966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:46.897532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 951/1942] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:46 2022] DELETE /identity/v3/users/cac587d6bcae46f7bc4d086fd6a5c70b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:46.898529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:46.900556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:46.981151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a6a73-cc79-47a8-9a27-f7ba85ea1789 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:46.988777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:46.990160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:46.990160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:46.990868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:46.992937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:47.056489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 992/1943] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:46 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:47.056489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.056871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:47.069804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.071244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1607e2-53bc-489e-8d8e-a61207751e09 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.071392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.085869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 952/1944] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:47.086068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.089824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.090826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.091245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.092143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] PATH_INFO: `/v3/users/7ea1b3c6937a49cca4b1ff398bd9cf5b` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.093799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea1b3c6937a49cca4b1ff398bd9cf5b)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.096523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.097381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c0caf-d26a-4316-8a3f-4f1a81b444dd service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.103222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.103915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.104317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.105213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.105603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.107741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.123176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 953/1945] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:47.127718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.140743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5937a-90e4-4c1d-91ab-33f292e4cc7c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.148216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.149141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.149417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.149701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] PATH_INFO: `/v3/projects/e3a2975d8ccd405e82dbf300972dd384/users/3de9f36f0da6484e8dbe1745a1d9909f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.196192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a2975d8ccd405e82dbf300972dd384, user_id=3de9f36f0da6484e8dbe1745a1d9909f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.199090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.388916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 954/1946] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:47 2022] PUT /identity/v3/projects/e3a2975d8ccd405e82dbf300972dd384/users/3de9f36f0da6484e8dbe1745a1d9909f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:47.393956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.474957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4718d7-55f4-4104-9426-2880d7c17c62 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.479217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea1b3c6937a49cca4b1ff398bd9cf5b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.489181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea1b3c6937a49cca4b1ff398bd9cf5b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.495473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.499853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.502260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.502475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.503597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.505683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.512574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea1b3c6937a49cca4b1ff398bd9cf5b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.513899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea1b3c6937a49cca4b1ff398bd9cf5b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.528318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 955/1947] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:47.528318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.528318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:47.544343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed5b8-a585-4b35-8b88-ec2931384842 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.551212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.556146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.556146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.556146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] PATH_INFO: `/v3/projects/7c254132b4d847d4a42292b731356447` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.556146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c254132b4d847d4a42292b731356447)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.556806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.645849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea1b3c6937a49cca4b1ff398bd9cf5b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.653671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 993/1948] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:47 2022] DELETE /identity/v3/users/7ea1b3c6937a49cca4b1ff398bd9cf5b => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:47.654762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.656370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:47.733822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c254132b4d847d4a42292b731356447', 'request_id': 'req-cc7638ce-826d-4a7f-8941-d45cf9b84d22'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.746139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c254132b4d847d4a42292b731356447', 'request_id': 'req-cc7638ce-826d-4a7f-8941-d45cf9b84d22'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.754573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c254132b4d847d4a42292b731356447', 'request_id': 'req-cc7638ce-826d-4a7f-8941-d45cf9b84d22'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.775266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a49a6-2884-44af-a56e-0dedc8227b97 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:47.914447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:47.916143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.916598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.917114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] PATH_INFO: `/v3/projects/e3a2975d8ccd405e82dbf300972dd384/users/3de9f36f0da6484e8dbe1745a1d9909f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:47.920685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a2975d8ccd405e82dbf300972dd384, user_id=3de9f36f0da6484e8dbe1745a1d9909f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:47.925871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:47.946241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 994/1949] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:47 2022] PUT /identity/v3/projects/e3a2975d8ccd405e82dbf300972dd384/users/3de9f36f0da6484e8dbe1745a1d9909f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:47.946241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:47.946241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:47.949253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] The token cache is being invalidate because project 7c254132b4d847d4a42292b731356447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:47.949253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.949253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cc7638ce-826d-4a7f-8941-d45cf9b84d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:47.955459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 956/1950] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:47 2022] DELETE /identity/v3/projects/7c254132b4d847d4a42292b731356447 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:47.961078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696a791-74aa-42d1-b175-1f9d1b1af581 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:47.963155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696a791-74aa-42d1-b175-1f9d1b1af581 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:47.963155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696a791-74aa-42d1-b175-1f9d1b1af581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.024279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:48.038159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2696a791-74aa-42d1-b175-1f9d1b1af581 None None] MFA Rules not processed for user `3de9f36f0da6484e8dbe1745a1d9909f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:48.039681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2696a791-74aa-42d1-b175-1f9d1b1af581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.054250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58457f3-c793-4df7-83bb-e70a8c7a5052 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.070198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.071099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.071435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.071682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] PATH_INFO: `/v3/projects/9f158a09f36d4393bcb1bf93b3db8fc7` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.080739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f158a09f36d4393bcb1bf93b3db8fc7)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.082711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.129748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 957/1951] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:48.131595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.132194 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.132486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.153931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:48.155354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.179687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 958/1952] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:48.180742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:48.182183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.221627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f158a09f36d4393bcb1bf93b3db8fc7', 'request_id': 'req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:48.250417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f158a09f36d4393bcb1bf93b3db8fc7', 'request_id': 'req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:48.310484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d2c36-87ae-4892-9ce5-c430d995f2fc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.376559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f158a09f36d4393bcb1bf93b3db8fc7', 'request_id': 'req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:48.379459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.384723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.385019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.385296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.394960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] The token cache is being invalidate because project 9f158a09f36d4393bcb1bf93b3db8fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:48.395321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:48.396019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5fbf1561-f6d1-453e-a589-a4ec1e613ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:48.399050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 995/1953] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:47 2022] DELETE /identity/v3/projects/9f158a09f36d4393bcb1bf93b3db8fc7 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:48.400998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.401414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.401899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.428347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:48.430142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.527903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.531487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.547092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 959/1954] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:48.548466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:48.550482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.590634 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 996/1955] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:48.595778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:48.604682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d7b70-8121-46bc-9c43-1ecad9bd0417 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.613671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.613671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.613671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.613671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.614125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:48.614125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.616071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.635776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 997/1956] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:48.640665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:48.656637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6799ea2-508d-4610-b0f4-3e2a97268f6d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.663048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.663886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.664343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.664767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.665760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.667977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.705011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb763e7-c7d1-48dd-90db-d73f27fdc9c3 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.710384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.711335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.712747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.712747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] PATH_INFO: `/v3/users/2699763c94b34212a73788b41c039340` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.713305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2699763c94b34212a73788b41c039340)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.715323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.777394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 998/1957] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:48 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:48.777394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:48.777394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.836936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:48.844390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41129-130b-4759-ab4b-df0aca949bcc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:48.848951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:48.849618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:48.849904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:48.850183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:48.851982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:48.920532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:48.922042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:48.932322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 999/1958] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:48.933143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.009977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768e49-a984-4c4e-9b2e-1bb84b7b606e service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.016586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.017753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.018347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.018667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.020229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.022688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.023360 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2699763c94b34212a73788b41c039340'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.024442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2699763c94b34212a73788b41c039340'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.053310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2699763c94b34212a73788b41c039340'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.061605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2699763c94b34212a73788b41c039340'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.149976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1000/1959] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:48 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:49.158813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.158813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.166999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2a205-a9ca-490a-9c1a-92ad8de30804 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.172276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.173667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.173950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.174231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] PATH_INFO: `/v3/projects/49c1f6b87632493db7de3f7cceba77ba` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.175467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c1f6b87632493db7de3f7cceba77ba)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.177384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.182940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2699763c94b34212a73788b41c039340'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.191493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 960/1960] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:48 2022] DELETE /identity/v3/users/2699763c94b34212a73788b41c039340 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:49.192361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.193775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.275383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf15c3a-bc15-46fe-b2c7-ae503f3fbc8c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.280164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.280859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.281150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.281461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.282210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.286264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.306355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 961/1961] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:49.311079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.325314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d80f-067e-41c5-9ab1-41a7efe65680 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.332801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.333736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.334079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.354199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.354199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.354199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.359187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 962/1962] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:49.363484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.404389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf282d1-ed03-4049-8266-ee77389330bb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.408543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c1f6b87632493db7de3f7cceba77ba', 'request_id': 'req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.415429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c1f6b87632493db7de3f7cceba77ba', 'request_id': 'req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.481285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.482228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.482516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.482802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] PATH_INFO: `/v3/projects/a2431cf8c26a4c9a8dffd0dc62831631/users/cb54f74f2fd74f09b6424bc6c81f04a1/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.497770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c1f6b87632493db7de3f7cceba77ba', 'request_id': 'req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.502143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2431cf8c26a4c9a8dffd0dc62831631, user_id=cb54f74f2fd74f09b6424bc6c81f04a1, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.505484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683493e-1dc6-4160-826b-2e2a49cafb36 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.529797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] The token cache is being invalidate because project 49c1f6b87632493db7de3f7cceba77ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:49.530163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.531409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f3ff5269-ffb1-46a5-af44-aa77f6d4d234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:49.535169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1001/1963] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:49 2022] DELETE /identity/v3/projects/49c1f6b87632493db7de3f7cceba77ba => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:49.540352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1badb5f-9dcf-4ac4-967d-926c54daaa4e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.540892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1badb5f-9dcf-4ac4-967d-926c54daaa4e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.541352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1badb5f-9dcf-4ac4-967d-926c54daaa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.568876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a1badb5f-9dcf-4ac4-967d-926c54daaa4e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:49.572241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a1badb5f-9dcf-4ac4-967d-926c54daaa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.606908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1002/1964] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:49.615476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.615777 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.616045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.639022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:49.649492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.650466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1003/1965] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:49.655082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.667440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b72d-ed8e-4c44-8cd8-ac0e93527d48 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.674255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.675285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.675818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.676181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.676785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:49.677102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.678932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.695189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1004/1966] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:17:49.697398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.708007 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229c3-a286-418b-a133-c2e55f38992f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.713061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.713833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.714148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.714459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.715299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:49.717235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:49.755373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 963/1967] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:49 2022] PUT /identity/v3/projects/a2431cf8c26a4c9a8dffd0dc62831631/users/cb54f74f2fd74f09b6424bc6c81f04a1/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:49.760643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.760933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.761232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.782394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:49.783751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.849856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 964/1968] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:49.850724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.853834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.911085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c77ec-4fa4-4066-8c9a-e7961e7e5299 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:49.914888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:49.915575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:49.915860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:49.916138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:49.920828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:49.965727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1005/1969] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:49 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:49.965727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:49.967763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:50.019345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.022564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.054008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 965/1970] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:50.054706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:50.055869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:50.072314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26621b8-fd5f-496d-8959-d8dc43a0276c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.076520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.078389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.078757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.079264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.092112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2c723-6082-4cc1-ad26-e4c70c8ec048 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.244068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.245140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.246925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.248009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] PATH_INFO: `/v3/projects/2327af55bb0a4ad2ae8817a266b75425` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.250357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2327af55bb0a4ad2ae8817a266b75425)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.256889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.274669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.276756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.291110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1006/1971] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:50.292348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:50.306364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51711ae-d1a2-4ef2-9bb7-ebc9424f74de service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.314906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.316489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.316616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.316860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.317604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.321029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.470717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2327af55bb0a4ad2ae8817a266b75425', 'request_id': 'req-5d1f3298-eb08-4aab-89c4-2ae581531c25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:50.479034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2327af55bb0a4ad2ae8817a266b75425', 'request_id': 'req-5d1f3298-eb08-4aab-89c4-2ae581531c25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:50.491211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2327af55bb0a4ad2ae8817a266b75425', 'request_id': 'req-5d1f3298-eb08-4aab-89c4-2ae581531c25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:50.543156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1007/1972] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:50 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:50.545669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:50.561474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697abf-c4fa-4f21-861f-7e4dca171ff6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.645019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.645662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.645943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.646218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.646953 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.648635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.659929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1008/1973] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:50.663778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:50.675167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cc57-3a25-4265-9d03-139a9faeb3c3 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.679993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.680704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.680991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.681276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.682311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.684036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.694981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1009/1974] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:50.715730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:50.728364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd378dac-3c31-4c70-ad19-c38de73317b0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:50.734480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:50.736219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.736730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.737075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] PATH_INFO: `/v3/projects/4f4c143f42cf4babaca91135c6fe7817/users/34ad740efe5c45dd9b821b3aa6482a6b/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.755329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4c143f42cf4babaca91135c6fe7817, user_id=34ad740efe5c45dd9b821b3aa6482a6b, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:50.758784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59480-0053-4f7f-93da-9754167277f9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:50.840380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] The token cache is being invalidate because project 2327af55bb0a4ad2ae8817a266b75425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:50.840759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:50.841477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5d1f3298-eb08-4aab-89c4-2ae581531c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:50.843221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 966/1975] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:50 2022] DELETE /identity/v3/projects/2327af55bb0a4ad2ae8817a266b75425 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:50.848337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00952-e50b-44e2-9704-f1d2a955a170 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.848631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00952-e50b-44e2-9704-f1d2a955a170 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.848920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00952-e50b-44e2-9704-f1d2a955a170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.864018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1010/1976] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:50 2022] PUT /identity/v3/projects/4f4c143f42cf4babaca91135c6fe7817/users/34ad740efe5c45dd9b821b3aa6482a6b/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:50.880574 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:50.881668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:50.882255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:50.882483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dfe00952-e50b-44e2-9704-f1d2a955a170 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:50.883877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dfe00952-e50b-44e2-9704-f1d2a955a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:50.925256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] MFA Rules not processed for user `34ad740efe5c45dd9b821b3aa6482a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:50.927024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:51.009404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 967/1977] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:51.011217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.011520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.011829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.031209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1011/1978] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:51.040469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:51.041829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:51.048438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.048438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:51.055460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 968/1979] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:51.060088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.070684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88298-8897-4eca-8bb7-9067a98acaf8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.075931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.076622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.076909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.077186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.078287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:51.078524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.080888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.094528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 969/1980] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:17:51.099344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.109628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:51.109858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ed91c-7fc5-44b1-9daa-8e5b84ffc0d6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.121682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.122084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.123911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.123911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.123911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.125426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.164457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3c27d-d607-4f2b-818e-f4973c271acf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.171181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.171937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.172196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.172679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.195217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.196880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.207888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1012/1981] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:51.297514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 970/1982] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:51 2022] POST /identity/v3/projects => generated 407 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:51.299154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.311509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbcb2f-6a3d-499e-afd4-bea02704f182 service glance] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.424414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.425562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.425905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.426223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.428072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.431595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.583693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1013/1983] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:51 2022] POST /identity/v3/users => generated 304 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:51.588079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.601106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513abc06-5fb1-4635-beed-6c2069c5d673 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.613605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.614663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.615219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.615671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.636509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.636509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.636509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 971/1984] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:51.639858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.665645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33ee9-ca10-4b76-94bf-4519c5d45e57 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.671342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.672759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.673622 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.673894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.674933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.679303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854a8f-aad6-4d90-8ad2-317f6fabc915 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.689352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1014/1985] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:51.693686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:51.731034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448172c-6562-43c4-b995-a583c8c54fd2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:51.746727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:51.747674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.747965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.748442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] PATH_INFO: `/v3/projects/b6460cf8d0e84e8290a211353bf3d4dc/users/3795a48608734b308063fe77ab25b757/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.785516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6460cf8d0e84e8290a211353bf3d4dc, user_id=3795a48608734b308063fe77ab25b757, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:51.790129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:51.917335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 972/1986] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:51 2022] PUT /identity/v3/projects/b6460cf8d0e84e8290a211353bf3d4dc/users/3795a48608734b308063fe77ab25b757/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:51.928276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:51.928570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:51.928929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:51.942514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:51.961226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] MFA Rules not processed for user `3795a48608734b308063fe77ab25b757`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:51.962477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:52.045970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:52.050306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1015/1987] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:52.055893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.122594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c289-001d-4b47-a152-c7542cdd928c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.128648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.129348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.129651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.129943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.130691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.132806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db78dab-0ba6-49e2-8c66-4889685667c8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.355240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 973/1988] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:52 2022] POST /identity/v3/projects => generated 407 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:52.356613 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.368081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda07b65-3b95-4ab8-8050-c1668380cf59 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.435865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.436097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.436367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.443916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.444610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.444897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.445211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.445868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.447831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.466590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] MFA Rules not processed for user `cb54f74f2fd74f09b6424bc6c81f04a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:52.467953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:52.589304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 974/1989] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:52.593887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.595932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:52.615623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfb24-35ee-4885-b824-cfab7ee5a876 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.627819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1016/1990] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:52 2022] POST /identity/v3/users => generated 310 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:52.632337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.635290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.636262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.636677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.636969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.637808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.639850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.644231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6164-85ce-4314-98c6-9b421d6cec88 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.650119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.651294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.651600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.651913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.653102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.655366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.667094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1017/1991] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:52.671346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.682430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660934-1d6d-4de4-a1d8-2484a599aef2 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.718168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.718168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.718168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.718168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] PATH_INFO: `/v3/projects/87ffeb889c1f4434960754faf9be955d/users/8094a3fe54c345a08fa6d3f0f46650d5/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.720127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ffeb889c1f4434960754faf9be955d, user_id=8094a3fe54c345a08fa6d3f0f46650d5, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.720127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.813113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 975/1992] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:52 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:52.819380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.832657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1018/1993] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:52 2022] PUT /identity/v3/projects/87ffeb889c1f4434960754faf9be955d/users/8094a3fe54c345a08fa6d3f0f46650d5/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:52.839224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.920141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3535-08b3-4dbe-819d-739f3b85c8f9 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.927187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.928228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.928581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.929106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.929990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.931894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.947042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1019/1994] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:52.956169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c659-e264-45e1-a692-2c34c5b02747 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:52.976161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:52.977854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:52.978590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:52.978880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:52.979186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:52.979919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:52.981925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:52.989519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3cb1-32a7-43d0-95ae-3a85c24785b6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.014727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.015748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.016232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.017047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] PATH_INFO: `/v3/projects/87ffeb889c1f4434960754faf9be955d/users/8094a3fe54c345a08fa6d3f0f46650d5/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.028556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ffeb889c1f4434960754faf9be955d, user_id=8094a3fe54c345a08fa6d3f0f46650d5, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.031550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249cb91-8e8f-4dac-b2f9-f7886d033358 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.066093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1020/1995] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:52 2022] PUT /identity/v3/projects/87ffeb889c1f4434960754faf9be955d/users/8094a3fe54c345a08fa6d3f0f46650d5/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:53.073259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.073697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.073972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.121281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] MFA Rules not processed for user `8094a3fe54c345a08fa6d3f0f46650d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:53.124092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:53.133670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 976/1996] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:52 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:53.137098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.170858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258668-6cc1-47a7-970a-29ba4a587dcf admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.255218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1021/1997] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:53.267289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.268983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:53.372623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.374169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.374466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.374794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.376353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.384181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19557-b2fb-4e77-abac-e171722002f6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.385155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.385273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.385273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.385442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.389263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.392116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 977/1998] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:53.396508 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.402263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.404034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.416538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34f9cf-7346-418d-bb55-cc60601bac35 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.422253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.424052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1022/1999] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:53 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:53.424510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.424807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.425092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] PATH_INFO: `/v3/projects/1e6ac66f61084132bb7f30b39245afbc/users/31c6117ef2f343e8b7b329e5dede13d0/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.455793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6ac66f61084132bb7f30b39245afbc, user_id=31c6117ef2f343e8b7b329e5dede13d0, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.458546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.572521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 978/2000] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:53 2022] PUT /identity/v3/projects/1e6ac66f61084132bb7f30b39245afbc/users/31c6117ef2f343e8b7b329e5dede13d0/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:53.576864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.589775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10d9ae-2e73-4cfb-b5f2-a9ba57498902 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.693002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.693835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.695045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.695045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.695290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.696805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.708692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1023/2001] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:53.713612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.726905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496cd30-6dc5-41d3-b75c-049ca1eb2539 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:53.732521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:53.733612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:53.733905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:53.734193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] PATH_INFO: `/v3/projects/1e6ac66f61084132bb7f30b39245afbc/users/31c6117ef2f343e8b7b329e5dede13d0/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:53.736882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6ac66f61084132bb7f30b39245afbc, user_id=31c6117ef2f343e8b7b329e5dede13d0, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:53.742146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:53.753233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 979/2002] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:53 2022] PUT /identity/v3/projects/1e6ac66f61084132bb7f30b39245afbc/users/31c6117ef2f343e8b7b329e5dede13d0/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:53.880262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:53.988282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0e1cb-4b04-4720-951d-d1696ba5339e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:54.043085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:54.043840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:54.044070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:54.044347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:54.137578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:54.139899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777e783-0376-419d-bf3c-04be9c04548d service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:54.164663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1024/2003] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:54.263672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:54.267764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:54.274514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:54.355107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:54.418275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddacfd-c0f5-483d-a148-5feb86a25ec3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:54.426319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:54.427553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:54.427844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:54.429799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] PATH_INFO: `/v3/users/4a4323ceb232418ea76f01545b92571b` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:54.430489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4323ceb232418ea76f01545b92571b)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:54.432869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:54.722275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4323ceb232418ea76f01545b92571b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:54.723715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4323ceb232418ea76f01545b92571b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:54.751471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4323ceb232418ea76f01545b92571b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:54.761438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4323ceb232418ea76f01545b92571b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:54.871819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4323ceb232418ea76f01545b92571b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:54.888844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 980/2004] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:54 2022] DELETE /identity/v3/users/4a4323ceb232418ea76f01545b92571b => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:54.894402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0f1b-bb03-4d64-b15b-5f3af063614d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:54.894980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0f1b-bb03-4d64-b15b-5f3af063614d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:54.894980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0f1b-bb03-4d64-b15b-5f3af063614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:54.930906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-fb3e0f1b-bb03-4d64-b15b-5f3af063614d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:54.933176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fb3e0f1b-bb03-4d64-b15b-5f3af063614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:55.036019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1025/2005] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:55.053767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:55.057477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:55.177478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75108900-2d1b-4a3e-acf4-219edcebd51e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:55.185720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:55.186958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:55.187382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:55.187813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:55.205531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:55.208957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:55.230368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 981/2006] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:55.947840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:55.948713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:55.949159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:56.027041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] MFA Rules not processed for user `31c6117ef2f343e8b7b329e5dede13d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:56.028316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:56.128265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:56.135857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:56.165327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1026/2007] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:56.175889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02eb7-4aa5-49c9-9c51-3f93f49a126d service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:56.196296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:56.198078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:56.206759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:56.207697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:56.207974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:56.209451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] PATH_INFO: `/v3/projects/25d02814ccef4b4cb90db19c0eb21ee4` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:56.212545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d02814ccef4b4cb90db19c0eb21ee4)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:56.216304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:56.319075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:56.330852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4e9d8-4d67-40f7-87c4-71338ef3a455 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:56.337986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:56.339112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:56.339767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:56.340162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:56.343236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:56.450587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:56.452529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:56.463369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1027/2008] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:56.486152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:56.508791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d02814ccef4b4cb90db19c0eb21ee4', 'request_id': 'req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:56.516978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d02814ccef4b4cb90db19c0eb21ee4', 'request_id': 'req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:56.525967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d02814ccef4b4cb90db19c0eb21ee4', 'request_id': 'req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:56.657934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] The token cache is being invalidate because project 25d02814ccef4b4cb90db19c0eb21ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:56.658352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:56.659157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:56.660915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 982/2009] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:56 2022] DELETE /identity/v3/projects/25d02814ccef4b4cb90db19c0eb21ee4 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:56.666072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:56.678787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4242a86-cdf3-485a-8548-3ba46dc36f85 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:56.758727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:56.760222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:56.760677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:56.761089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] PATH_INFO: `/v3/users/3de9f36f0da6484e8dbe1745a1d9909f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:56.762998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de9f36f0da6484e8dbe1745a1d9909f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:56.766776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:57.072243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de9f36f0da6484e8dbe1745a1d9909f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.084018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de9f36f0da6484e8dbe1745a1d9909f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.119036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de9f36f0da6484e8dbe1745a1d9909f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.119827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de9f36f0da6484e8dbe1745a1d9909f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.225368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:57.227243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:57.250040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c946c6-4ddd-4b71-b8b7-5f31784e8d14 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:57.257447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:57.258413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:57.258702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:57.258987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] PATH_INFO: `/v3/users/874042de57cb4e919f995ecb287d1ff2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:57.260282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874042de57cb4e919f995ecb287d1ff2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:57.262369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:57.274887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-d280d04d-8b8d-4aba-8abc-50e6bccfb785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de9f36f0da6484e8dbe1745a1d9909f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.288718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1028/2010] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:56 2022] DELETE /identity/v3/users/3de9f36f0da6484e8dbe1745a1d9909f => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:57.570489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874042de57cb4e919f995ecb287d1ff2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.571786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874042de57cb4e919f995ecb287d1ff2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.591952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874042de57cb4e919f995ecb287d1ff2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.610479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874042de57cb4e919f995ecb287d1ff2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.703003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874042de57cb4e919f995ecb287d1ff2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:57.715731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 983/2011] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:57 2022] DELETE /identity/v3/users/874042de57cb4e919f995ecb287d1ff2 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:57.720964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982345-334e-45cc-a624-358553e8076f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:57.721403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982345-334e-45cc-a624-358553e8076f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:57.721897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982345-334e-45cc-a624-358553e8076f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:57.741873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4a982345-334e-45cc-a624-358553e8076f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:57.743113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4a982345-334e-45cc-a624-358553e8076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:57.847277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1029/2012] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:17:57.856502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:57.858031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:57.930467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a982345-334e-45cc-a624-358553e8076f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:57.934221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4a982345-334e-45cc-a624-358553e8076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:57.942605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d480eb-fe4a-4f15-a7dd-0c3616afc1b4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:57.948619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:57.949610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:57.950025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:57.950316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:57.964652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a982345-334e-45cc-a624-358553e8076f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:57.967263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:57.968805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ce9ca-a4f7-4517-92d1-467b08b15226 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:57.971698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:57.981377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 984/2013] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:17:57.985633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:57.986302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:57.986814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] PATH_INFO: `/v3/projects/e3a2975d8ccd405e82dbf300972dd384` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:57.990258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a2975d8ccd405e82dbf300972dd384)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:57.990764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:58.170950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a2975d8ccd405e82dbf300972dd384', 'request_id': 'req-72513268-a5f9-424c-b152-8b55b865096d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.190180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a2975d8ccd405e82dbf300972dd384', 'request_id': 'req-72513268-a5f9-424c-b152-8b55b865096d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.313658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a2975d8ccd405e82dbf300972dd384', 'request_id': 'req-72513268-a5f9-424c-b152-8b55b865096d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.339917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] The token cache is being invalidate because project e3a2975d8ccd405e82dbf300972dd384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:58.340411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.341537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-72513268-a5f9-424c-b152-8b55b865096d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.346508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1030/2014] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:57 2022] DELETE /identity/v3/projects/e3a2975d8ccd405e82dbf300972dd384 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:58.353314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:58.354983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:58.355604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:58.361015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:58.402924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:58.404747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:58.524776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 985/2015] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:58.536186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:58.536557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:58.536937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:58.593608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:58.597732 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:58.599964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:58.628229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1031/2016] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:58.633516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:58.645017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7bfa1-6d58-4180-95c8-4136938c20fa None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:58.650676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:58.651722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:58.651980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:58.652258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:58.653171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:17:58.653481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:58.655100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:58.674482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 986/2017] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:17:58.686074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:58.704291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12c868-cfc4-477d-b907-7db806c5c598 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:58.713813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:58.713813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:58.714228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:58.714973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:58.716059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:58.720213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:58.818267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:58.821324 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:58.839590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718bf4c-b888-4f93-8bb4-087646d86797 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:58.845329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:58.846506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:58.846729 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:58.847059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] PATH_INFO: `/v3/projects/67aaaf0063044155a845e44eea2dcad4` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:58.848579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=67aaaf0063044155a845e44eea2dcad4)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:58.852320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:58.861406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1032/2018] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:58 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:17:58.866659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:58.976454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9325f5-4f93-49b9-89e8-bddb530b35af admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:58.978988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67aaaf0063044155a845e44eea2dcad4', 'request_id': 'req-58b1cd76-c102-4e30-be41-2fde852a7ffe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.986816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67aaaf0063044155a845e44eea2dcad4', 'request_id': 'req-58b1cd76-c102-4e30-be41-2fde852a7ffe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:58.995536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67aaaf0063044155a845e44eea2dcad4', 'request_id': 'req-58b1cd76-c102-4e30-be41-2fde852a7ffe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.053450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.054224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.054514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.054796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.055939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.057986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.103173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] The token cache is being invalidate because project 67aaaf0063044155a845e44eea2dcad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:17:59.103526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.104326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.106037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 987/2019] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:17:58 2022] DELETE /identity/v3/projects/67aaaf0063044155a845e44eea2dcad4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:59.113968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.116329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.148471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1cd76-c102-4e30-be41-2fde852a7ffe admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.154625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.155695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.156043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.156435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] PATH_INFO: `/v3/users/e561baf4add24569a23922f81b6fbcf1` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.158472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e561baf4add24569a23922f81b6fbcf1)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.160809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.197614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1033/2020] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:17:58 2022] POST /identity/v3/users => generated 291 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:17:59.202245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.209457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.224547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd5f0-3f53-474a-8884-889a0d903d51 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.232804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.234170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.235087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.235778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.237718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.241247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.268963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1034/2021] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:17:59.273793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.293135 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2f3fc-09cf-41f8-b8ad-a15df4e31a3a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.304558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.305804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.306363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.306887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.308981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.312124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.333751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1035/2022] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:17:59.338721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.359882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fd963-0412-4865-8183-ac7d75bd5375 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.378377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.379993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.380476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.381068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] PATH_INFO: `/v3/projects/fdfeffe85c4c4991b2cae637e5cdb641/users/2925ea69f6d946dda061bfd8ff362ad2/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.409688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdfeffe85c4c4991b2cae637e5cdb641, user_id=2925ea69f6d946dda061bfd8ff362ad2, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.410191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ff12b-2533-456d-8064-03e318a3a9e7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.495291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e561baf4add24569a23922f81b6fbcf1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.496163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e561baf4add24569a23922f81b6fbcf1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.519789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e561baf4add24569a23922f81b6fbcf1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.520709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1036/2023] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:17:59 2022] PUT /identity/v3/projects/fdfeffe85c4c4991b2cae637e5cdb641/users/2925ea69f6d946dda061bfd8ff362ad2/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:17:59.538069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.538381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.538663 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.540814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e561baf4add24569a23922f81b6fbcf1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.574957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] MFA Rules not processed for user `2925ea69f6d946dda061bfd8ff362ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:17:59.576560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.647989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e561baf4add24569a23922f81b6fbcf1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:17:59.658159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 988/2024] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:17:59 2022] DELETE /identity/v3/users/e561baf4add24569a23922f81b6fbcf1 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:17:59.669631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.679536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.706134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1037/2025] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:17:59.712249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:59.713769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.819748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94aae8-c13b-43a5-be3a-3cec01f3f293 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.824856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.826248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.826248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.826248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.830281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:59.833888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ec31-8601-43ff-92ba-67a6d864e805 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:59.980845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:59.981726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:59.982013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:59.982299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:59.983015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:59.988307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:59.997051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.007300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.024905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 989/2026] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:00.118602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1038/2027] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:17:59 2022] POST /identity/v3/projects => generated 383 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:00.124361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.146853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ac70c-1586-418b-b2a6-98d5e96e2528 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.263780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:00.298019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:00.301603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.303015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.303709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.304266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:00.305563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.309560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.351347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.354182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:00.394394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd1514-27e7-4390-8b17-cb216cda245e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.403380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.405589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.405589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.406059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] PATH_INFO: `/v3/projects/40d1f3d2afd646eb88d645cde8b6dc30` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:00.409038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d1f3d2afd646eb88d645cde8b6dc30)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.411203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.425245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 990/2028] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:00 2022] POST /identity/v3/users => generated 298 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:00.432553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.455892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4dee7-8295-471c-b954-d6cb7ca75488 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.463998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.465285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.465894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.467016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:00.468479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.473332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.490790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 991/2029] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:00.498006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.517028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a0927-6b72-40d9-b6ca-64f3b93b0060 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.525115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.526741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.527291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.527776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] PATH_INFO: `/v3/projects/f4180fe05b864430ac05a88943f445b0/users/f1dfc8bab21a45e5bd07928f105cab3a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:00.553656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4180fe05b864430ac05a88943f445b0, user_id=f1dfc8bab21a45e5bd07928f105cab3a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.556573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.793902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 992/2030] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:00 2022] PUT /identity/v3/projects/f4180fe05b864430ac05a88943f445b0/users/f1dfc8bab21a45e5bd07928f105cab3a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:00.798930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.810216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d1f3d2afd646eb88d645cde8b6dc30', 'request_id': 'req-46670a5a-e1fe-4336-acf3-5463679bad0a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:00.823116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d1f3d2afd646eb88d645cde8b6dc30', 'request_id': 'req-46670a5a-e1fe-4336-acf3-5463679bad0a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:00.873918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c17ab4-f957-459f-b9b7-06c243f3dfd6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.946906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.947631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.947920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.948207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:00.948942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:00.950623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:00.961510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 993/2031] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:00.969867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:00.981724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1d6a-deff-445d-b73f-6a6637dacda8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:00.982716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d1f3d2afd646eb88d645cde8b6dc30', 'request_id': 'req-46670a5a-e1fe-4336-acf3-5463679bad0a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:00.994216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:00.996080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:00.996488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:00.996896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] PATH_INFO: `/v3/projects/f4180fe05b864430ac05a88943f445b0/users/f1dfc8bab21a45e5bd07928f105cab3a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:01.002872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4180fe05b864430ac05a88943f445b0, user_id=f1dfc8bab21a45e5bd07928f105cab3a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:01.007660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9e6d4-6970-41f8-b458-9c0fa221566a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:01.010553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] The token cache is being invalidate because project 40d1f3d2afd646eb88d645cde8b6dc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:01.011181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.013095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.018901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1039/2032] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:00 2022] DELETE /identity/v3/projects/40d1f3d2afd646eb88d645cde8b6dc30 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:01.023315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 994/2033] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:00 2022] PUT /identity/v3/projects/f4180fe05b864430ac05a88943f445b0/users/f1dfc8bab21a45e5bd07928f105cab3a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:01.024269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:01.039512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:01.039654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:01.039654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:01.039654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:01.059354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46670a5a-e1fe-4336-acf3-5463679bad0a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:01.073733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] MFA Rules not processed for user `f1dfc8bab21a45e5bd07928f105cab3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:01.075005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:01.129173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:01.130376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:01.130376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:01.130709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] PATH_INFO: `/v3/users/71039acecc964457b3b3034dda7c9a65` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:01.132036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71039acecc964457b3b3034dda7c9a65)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:01.134168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:01.191791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 995/2034] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:01.211224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:01.212722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:01.286971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcccc5-1b5f-4506-a70f-0972d429bc3d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:01.291294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:01.292155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:01.292456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:01.293005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:01.303578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:01.306477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:01.325344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 996/2035] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:01.404085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71039acecc964457b3b3034dda7c9a65'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.414936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71039acecc964457b3b3034dda7c9a65'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.436451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71039acecc964457b3b3034dda7c9a65'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.437249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71039acecc964457b3b3034dda7c9a65'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.609317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:01.617130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71039acecc964457b3b3034dda7c9a65'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:01.620880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f8c21-d905-40a6-8f58-373b0285749e service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:01.624611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1040/2036] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:01 2022] DELETE /identity/v3/users/71039acecc964457b3b3034dda7c9a65 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:01.626201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:01.627042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:01.633895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:01.634263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:01.636887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:01.714529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:01.716033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:01.727886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 997/2037] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:02.109027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:02.111911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.131675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:02.225826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae05df-672d-4902-bbf1-f23f385b2fc1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:02.233690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:02.235816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.236411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.236801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] PATH_INFO: `/v3/projects/147905762c6d4c789b3a68d82b0a5cc7` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.239290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=147905762c6d4c789b3a68d82b0a5cc7)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:02.241402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:02.322423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:02.324227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.348689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6807c-d69c-4d2e-9cb2-96acf1f39cce service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:02.356439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:02.357935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.358430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.358993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] PATH_INFO: `/v3/users/2925ea69f6d946dda061bfd8ff362ad2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.361214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2925ea69f6d946dda061bfd8ff362ad2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:02.364614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:02.435001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147905762c6d4c789b3a68d82b0a5cc7', 'request_id': 'req-18aeca45-f341-42a2-820a-af22ed387fde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.443841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147905762c6d4c789b3a68d82b0a5cc7', 'request_id': 'req-18aeca45-f341-42a2-820a-af22ed387fde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.548590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147905762c6d4c789b3a68d82b0a5cc7', 'request_id': 'req-18aeca45-f341-42a2-820a-af22ed387fde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.568469 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] The token cache is being invalidate because project 147905762c6d4c789b3a68d82b0a5cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:02.568849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.569815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-18aeca45-f341-42a2-820a-af22ed387fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.572489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1041/2038] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:02 2022] DELETE /identity/v3/projects/147905762c6d4c789b3a68d82b0a5cc7 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:02.582807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12c440-a09f-48e3-a9b3-4dd66220f031 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.583321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12c440-a09f-48e3-a9b3-4dd66220f031 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.583773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12c440-a09f-48e3-a9b3-4dd66220f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.608542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2925ea69f6d946dda061bfd8ff362ad2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.609952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2925ea69f6d946dda061bfd8ff362ad2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.615433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3c12c440-a09f-48e3-a9b3-4dd66220f031 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:02.617542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3c12c440-a09f-48e3-a9b3-4dd66220f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.637008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2925ea69f6d946dda061bfd8ff362ad2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.645999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2925ea69f6d946dda061bfd8ff362ad2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.712614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1042/2039] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:02.726757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.727314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.727834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.741277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a7f074b2-0155-4746-a0d9-f939ddfacaf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2925ea69f6d946dda061bfd8ff362ad2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:02.752610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 998/2040] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:02 2022] DELETE /identity/v3/users/2925ea69f6d946dda061bfd8ff362ad2 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:02.754810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:02.756625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.758080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.760452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.760452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.780544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:02.783005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.811189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:02.838013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1043/2041] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:02.843045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:02.854170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ee1a3-a451-4aca-b50f-6f42d3da650e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:02.861698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:02.862594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.863096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.863693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.864951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:02.865406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 999/2042] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:02.865560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:02.868021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:02.880885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:02.882596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:02.889787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1044/2043] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:02.892095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:02.904114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf9826-f499-462d-b31b-1e0f4b527345 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:02.910884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:02.912022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.912477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.912842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.913635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:02.915877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:02.953770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e374551-1d35-4661-9ffb-01a64193d3ee None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:02.957908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:02.958657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:02.959053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:02.959430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:02.969962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:02.971464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:02.982123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1000/2044] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:03.029635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1045/2045] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:02 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:03.034273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.046386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece51b3-ab59-4e0a-8009-63b3908393eb service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.126295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.126960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.127291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.127572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.128279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.130360 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.147575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:03.207406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1001/2046] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:03 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:03.211814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.229331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f7d3a-b20d-4e9f-af91-05870bbdbc6f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.237518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.238553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.239032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.239611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.243274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.243274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.256202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1046/2047] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:03.261371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.274665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a67233-5325-4cd0-b5fe-db3ba76db6f3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.280819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.281696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.282145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.282546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.283529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.285737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26716e-c415-450c-9f96-092e1a4d56e2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.300580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1002/2048] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:03.305357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.317531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fdb1e-a335-48ab-9d01-7937cde61af1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.322985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.324026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.324495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.324864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] PATH_INFO: `/v3/projects/4b73048e43674902b47bf7e2f38388cf/users/ea3b131d8dfb442e8076c73989442490/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.344868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b73048e43674902b47bf7e2f38388cf, user_id=ea3b131d8dfb442e8076c73989442490, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.347583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.446067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1047/2049] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:03 2022] PUT /identity/v3/projects/4b73048e43674902b47bf7e2f38388cf/users/ea3b131d8dfb442e8076c73989442490/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:03.466688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.467235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.467737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.494362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.497210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:03.531665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] MFA Rules not processed for user `ea3b131d8dfb442e8076c73989442490`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:03.533616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:03.633060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa7c10-6292-4bb8-931d-a6b71489c3b3 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.639876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1003/2050] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:03.641079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.642270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.642747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.643280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] PATH_INFO: `/v3/projects/fdfeffe85c4c4991b2cae637e5cdb641` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.644684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.645339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdfeffe85c4c4991b2cae637e5cdb641)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.651773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.655795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74237c-565f-495e-acc0-875683a3a09f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.662216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.662941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.666417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.666417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.666417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.666417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:03.768268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1004/2051] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:03 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:03.778348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.820504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdfeffe85c4c4991b2cae637e5cdb641', 'request_id': 'req-3450346e-8cb7-4d4d-8ef2-433a37c54e29'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:03.828754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdfeffe85c4c4991b2cae637e5cdb641', 'request_id': 'req-3450346e-8cb7-4d4d-8ef2-433a37c54e29'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:03.874209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315dc2e-9100-4835-8190-e9d537b4a008 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:03.908223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdfeffe85c4c4991b2cae637e5cdb641', 'request_id': 'req-3450346e-8cb7-4d4d-8ef2-433a37c54e29'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:03.928484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] The token cache is being invalidate because project fdfeffe85c4c4991b2cae637e5cdb641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:03.928797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:03.929491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:03.931402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1048/2052] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:03 2022] DELETE /identity/v3/projects/fdfeffe85c4c4991b2cae637e5cdb641 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:03.935344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:03.936933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:03.951038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:03.951423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:03.951820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:03.952175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:03.953186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:03.955455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:04.027473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450346e-8cb7-4d4d-8ef2-433a37c54e29 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:04.033165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:04.034064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.034347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.034627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] PATH_INFO: `/v3/users/f1dfc8bab21a45e5bd07928f105cab3a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.035871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1dfc8bab21a45e5bd07928f105cab3a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:04.037943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:04.191692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1005/2053] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:03 2022] POST /identity/v3/users => generated 289 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:04.195366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:04.197365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:04.216461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584e639-cfbb-4312-b8e6-1152ac052387 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:04.216873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:04.217619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.217952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.218195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.219591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:04.221257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:04.232974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1006/2054] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:04.238646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:04.249454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a1224-2a70-4746-93e9-1f621a7d8fff admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:04.254407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:04.255270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.255631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.255976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.256843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:04.258632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:04.270536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1007/2055] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:04.331452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:04.342899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ac09d-0e8e-4854-ba65-a9870ec3f881 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:04.348777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:04.349745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.350038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.350316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] PATH_INFO: `/v3/projects/34203791a0d64f23b88cb3f4e47b52f8/users/4cfaf7f2446542b0a1c520771d2fc125/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.367022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34203791a0d64f23b88cb3f4e47b52f8, user_id=4cfaf7f2446542b0a1c520771d2fc125, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:04.369700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438c392-4c6a-4e25-931c-a76a291eff32 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:04.565087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:04.648640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1008/2056] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:04 2022] PUT /identity/v3/projects/34203791a0d64f23b88cb3f4e47b52f8/users/4cfaf7f2446542b0a1c520771d2fc125/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:04.658807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:04.674448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.675149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.675652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.676095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1dfc8bab21a45e5bd07928f105cab3a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:04.684563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1dfc8bab21a45e5bd07928f105cab3a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:04.698702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1dfc8bab21a45e5bd07928f105cab3a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:04.700855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1dfc8bab21a45e5bd07928f105cab3a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:04.710377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] MFA Rules not processed for user `4cfaf7f2446542b0a1c520771d2fc125`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:04.712418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:04.827463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1009/2057] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:04.827730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1dfc8bab21a45e5bd07928f105cab3a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:04.831560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:04.833302 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:04.837911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1049/2058] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:03 2022] DELETE /identity/v3/users/f1dfc8bab21a45e5bd07928f105cab3a => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:04.923702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5401a-1de5-4aa9-b825-92c3cf681ef0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.040106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.041335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.041335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.041636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.042323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.044311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.200329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1010/2059] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:04 2022] POST /identity/v3/projects => generated 377 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:05.200466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.225855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01292572-7494-4021-b7d8-f6fdab13b02a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.303333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.304320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.304599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.304890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.305674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.309640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.425799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1050/2060] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:05 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:05.430279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.442243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87465dfb-67da-4976-a783-5c7416cb83b3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.447392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.451271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.451271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.451271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.451271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.451914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.467169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1011/2061] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:05.472773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.478195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.480559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:05.486309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91af00-cba4-4379-9244-71ee22069c6b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.511433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.514151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.514520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.514759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261561-db79-4051-a5b0-f70827a433d6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.514915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] PATH_INFO: `/v3/projects/cd1589d915834f30878806fdf35cd981/users/b42bed9d9d2e4725976a3f0cb6dd24f9/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.523675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.525189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.525705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.526192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] PATH_INFO: `/v3/projects/f4180fe05b864430ac05a88943f445b0` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.528435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4180fe05b864430ac05a88943f445b0)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.532550 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.535982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1589d915834f30878806fdf35cd981, user_id=b42bed9d9d2e4725976a3f0cb6dd24f9, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.539043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.693692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1051/2062] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:05 2022] PUT /identity/v3/projects/cd1589d915834f30878806fdf35cd981/users/b42bed9d9d2e4725976a3f0cb6dd24f9/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:05.698126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.740976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4180fe05b864430ac05a88943f445b0', 'request_id': 'req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:05.748261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4180fe05b864430ac05a88943f445b0', 'request_id': 'req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:05.756236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4180fe05b864430ac05a88943f445b0', 'request_id': 'req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:05.784032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7db637-4037-47b5-9fb4-2bd590ddfadf admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.867446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.868131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.868416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.868700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.869821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.872364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.886058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1052/2063] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:05.891088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:05.907200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f76a1-7cbc-4178-a062-ebe204844bcb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:05.908106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] The token cache is being invalidate because project f4180fe05b864430ac05a88943f445b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:05.908587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:05.909708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ac774de-67b4-4aa7-a46d-ec4ded863b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:05.912929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1012/2064] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:05 2022] DELETE /identity/v3/projects/f4180fe05b864430ac05a88943f445b0 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:05.917819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:05.919617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.920114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.920423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b1238-0649-48c2-91de-4b1b1044732d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.920685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] PATH_INFO: `/v3/projects/cd1589d915834f30878806fdf35cd981/users/b42bed9d9d2e4725976a3f0cb6dd24f9/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.920925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b1238-0649-48c2-91de-4b1b1044732d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.921938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b1238-0649-48c2-91de-4b1b1044732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:05.925089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1589d915834f30878806fdf35cd981, user_id=b42bed9d9d2e4725976a3f0cb6dd24f9, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.930988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:05.932164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c85076-1a42-447b-9904-04e215370487 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.951926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1053/2065] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:05 2022] PUT /identity/v3/projects/cd1589d915834f30878806fdf35cd981/users/b42bed9d9d2e4725976a3f0cb6dd24f9/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:05.958320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4c2b1238-0649-48c2-91de-4b1b1044732d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:05.963268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4c2b1238-0649-48c2-91de-4b1b1044732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:05.972428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:05.972898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:05.973331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.003002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] MFA Rules not processed for user `b42bed9d9d2e4725976a3f0cb6dd24f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:06.004716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:06.025718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:06.040170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1013/2066] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:06.053454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.053936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.055365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.092195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:06.093766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:06.111250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1014/2067] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:06.112831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1054/2068] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:06.118260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.129462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f258-ec17-4645-9127-e78cd2a42be9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.130756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.132904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:06.148130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.148130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.148741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.149221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.150650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:06.150996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.153048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.192039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1015/2069] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:06.198195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.216805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01f125-7d96-4ca4-b83d-551956a2fe49 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.231259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.232853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.232853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.233372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.236025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.239054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.239278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78b280-01ef-48c5-b076-b0cab29cff2a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.245493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.246309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.246793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.247375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.259713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:06.349465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1016/2070] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:06 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:06.353911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.434040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.436710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.455163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1055/2071] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:06.500922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bafdc-d8d5-4ac8-b8b5-00cf43487adc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.506914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.510943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.510943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.510943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.510943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.510943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.666459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1017/2072] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:06 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:06.670356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.693035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a8fa7-835f-46db-9cfb-317817409495 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.704025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.705518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.714100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.714100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.714100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.714100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.732795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1056/2073] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:06.734842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.749977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2d4dd-2553-4cb6-a5ba-b59739e49b15 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.756895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.758313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.758715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.759174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.760554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.763846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b5d6b-0e2b-4158-a6e1-09f3c57595f5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.791876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1018/2074] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:06.799410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.816467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e7c02-f24a-4afa-9241-6b9296903c40 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.822325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.823409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.823793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.824125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] PATH_INFO: `/v3/projects/976f72f4873540229a0ff788c89ebb8a/users/1e326eaff59445958ceddac5db96bd19/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.856054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=976f72f4873540229a0ff788c89ebb8a, user_id=1e326eaff59445958ceddac5db96bd19, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.860239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.994389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1057/2075] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:06 2022] PUT /identity/v3/projects/976f72f4873540229a0ff788c89ebb8a/users/1e326eaff59445958ceddac5db96bd19/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:07.014469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:07.015978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:07.015978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:07.055275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] MFA Rules not processed for user `1e326eaff59445958ceddac5db96bd19`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:07.056920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:07.147415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:07.179997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1019/2076] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:07.265563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:07.561212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:07.687759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:07.878153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:08.078398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e24702-fa0b-475d-802c-c6a35be628dc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:08.086000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:08.095549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:08.096697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:08.097206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:08.097693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:08.122239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:08.128618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:08.151113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1058/2077] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:08.318697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:08.420765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:09.240274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:09.487426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:09.965095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:09.978034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6457-c6cc-4967-bc9e-b9b436a6ec55 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:09.982164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:09.982845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:09.983174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:09.984096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:09.986414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:10.091152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:10.092655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:10.105350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1020/2078] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:10.144221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:10.290292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:10.879704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:11.019614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:11.490580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:11.617873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:11.869653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:12.127792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:12.183976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:12.364805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:12.512249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:12.913741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:12.928506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2660a-867c-46ee-bf65-a024b1a631f1 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:12.934658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:12.935800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:12.936332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:12.936764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:13.014816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:13.017312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a618f43-6d17-48b7-bf93-c4e1ba54d6ce service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:13.034052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1059/2079] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:13.206766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:13.316208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:13.461384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:13.786822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:13.815731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.426540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.536259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.538323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.590238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.626812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af677f5f-7efe-407d-94f2-7a43d82da624 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.631783 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.632676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.632957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.633236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] PATH_INFO: `/v3/users/3795a48608734b308063fe77ab25b757` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.634821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3795a48608734b308063fe77ab25b757)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.636642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.860562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3795a48608734b308063fe77ab25b757'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.861433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3795a48608734b308063fe77ab25b757'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.875842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3795a48608734b308063fe77ab25b757'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.885899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.888264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3795a48608734b308063fe77ab25b757'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.912751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.974990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3795a48608734b308063fe77ab25b757'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.986419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1021/2080] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:14 2022] DELETE /identity/v3/users/3795a48608734b308063fe77ab25b757 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:14.991974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.992295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.992576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.012863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:15.014139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.088406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1060/2081] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:15.100154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.101677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.110504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.136146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.188580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609b69-fc86-4590-ae2c-796897bef20b admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.193001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.193766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.194042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.194331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.210880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.213666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.228605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1022/2082] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:15.284441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.369090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.819603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.825658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.847591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16222d-d1db-4da8-9e6f-dced2f5887b5 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.860785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.861881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.862177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.862439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] PATH_INFO: `/v3/projects/b6460cf8d0e84e8290a211353bf3d4dc` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.863945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6460cf8d0e84e8290a211353bf3d4dc)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.866343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.159996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6460cf8d0e84e8290a211353bf3d4dc', 'request_id': 'req-0fee8a11-19c2-4566-b015-eb5364addc89'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.166824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6460cf8d0e84e8290a211353bf3d4dc', 'request_id': 'req-0fee8a11-19c2-4566-b015-eb5364addc89'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.267699 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6460cf8d0e84e8290a211353bf3d4dc', 'request_id': 'req-0fee8a11-19c2-4566-b015-eb5364addc89'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.288204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] The token cache is being invalidate because project b6460cf8d0e84e8290a211353bf3d4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:16.293403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.293403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.293403 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1061/2083] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:15 2022] DELETE /identity/v3/projects/b6460cf8d0e84e8290a211353bf3d4dc => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:16.297460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:16.309707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f05644-11b7-4207-a701-67f8ac5711cb service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:16.386149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:16.387434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.387857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.388261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] PATH_INFO: `/v3/users/8094a3fe54c345a08fa6d3f0f46650d5` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.390737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8094a3fe54c345a08fa6d3f0f46650d5)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:16.393810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.422275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:16.614542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8094a3fe54c345a08fa6d3f0f46650d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.615303 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8094a3fe54c345a08fa6d3f0f46650d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.638041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8094a3fe54c345a08fa6d3f0f46650d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.650122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8094a3fe54c345a08fa6d3f0f46650d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.741963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-253c43ce-895f-4e6b-a722-586f6d46bc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8094a3fe54c345a08fa6d3f0f46650d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.752393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1023/2084] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:16 2022] DELETE /identity/v3/users/8094a3fe54c345a08fa6d3f0f46650d5 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:16.765910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.215455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.312983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:17.315728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.317439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.338460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee8a11-19c2-4566-b015-eb5364addc89 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:17.408198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:17.408981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.409291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.409590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] PATH_INFO: `/v3/projects/87ffeb889c1f4434960754faf9be955d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.410921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ffeb889c1f4434960754faf9be955d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:17.413171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:17.483819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.581291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ffeb889c1f4434960754faf9be955d', 'request_id': 'req-3819a387-5f18-4392-a8d5-96d50b16091e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.589150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ffeb889c1f4434960754faf9be955d', 'request_id': 'req-3819a387-5f18-4392-a8d5-96d50b16091e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.666902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ffeb889c1f4434960754faf9be955d', 'request_id': 'req-3819a387-5f18-4392-a8d5-96d50b16091e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.692387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] The token cache is being invalidate because project 87ffeb889c1f4434960754faf9be955d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:17.692792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.696397 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.711606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3819a387-5f18-4392-a8d5-96d50b16091e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.711606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1062/2085] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:17 2022] DELETE /identity/v3/projects/87ffeb889c1f4434960754faf9be955d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:17.716374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.716895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.717488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.740999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:17.756329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.778064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.827936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1024/2086] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:17.835818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.836160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.836484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.861517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:17.863069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.871847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1063/2087] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:17.878061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:17.892154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d09acd-6c66-46e7-be36-e5c2be16c371 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:17.897944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:17.898652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.899025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.899304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.900096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:17.900443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:17.902161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:17.922073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1025/2088] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:17.927260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:17.942657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188ece6-0a73-462c-8aec-4ff5b07a0c09 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:17.948253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:17.949037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.949379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.949739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.950667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:17.953603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:17.967181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.052171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1064/2089] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:17 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:18.056858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.068648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0dce-9626-4440-99ce-2da2dc6c1eb2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.126021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.142098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.143055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.143460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.143743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.144477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.146809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.154272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.281047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1026/2090] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:18.284364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.303320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a74af5-f46d-4b4c-9e30-5252bf5597c4 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.319572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.320673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.321208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.321624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.322963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.326164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.352736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1065/2091] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:18.354392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.366727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26935c-eaa0-4242-ab62-cdb77e0e9e37 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.372772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.379767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.379767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.379767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.379767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.380435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.381913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacdc423-ad14-409e-be96-3f24577412b6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.397011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1027/2092] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:18.404068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.417200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21762ee3-1b32-4251-9d6b-1ff9eb9ce350 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.427294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.428916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.429407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.430032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] PATH_INFO: `/v3/projects/f8770f840b8c45328b96b76b6f09e9fe/users/5512aa89960645dabcdaba4d95d7cfa4/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.457902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8770f840b8c45328b96b76b6f09e9fe, user_id=5512aa89960645dabcdaba4d95d7cfa4, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.461626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.489711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.573529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1066/2093] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:18 2022] PUT /identity/v3/projects/f8770f840b8c45328b96b76b6f09e9fe/users/5512aa89960645dabcdaba4d95d7cfa4/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:18.588717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.589169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.589632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.638054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] MFA Rules not processed for user `5512aa89960645dabcdaba4d95d7cfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:18.640213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:18.667763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.724386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1028/2094] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:18.736649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.752974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.837255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcea70-e2e4-41e8-80b9-4b5bcdb668ff admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.847578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.848616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.849054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.849658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.850493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.850738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.853551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.971937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1067/2095] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:18.981534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.997205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b4257-5a03-4d7e-a76a-38042f215008 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.087216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.087929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.088216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.088496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.089220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.091181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.185617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1029/2096] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:19.190693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.202668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d5d16-f47c-42c7-a3da-67da131d2976 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.207399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.208064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.208359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.208636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.209373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.210970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.223293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1068/2097] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:19.227906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.239992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da93a0e-856c-4423-b319-96cbe046310b admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.246772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.247708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.248023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.248449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] PATH_INFO: `/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c/users/94bd5e48a65f4989bd0d69ef7351ef02/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.265135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e3b545c3fe4590ab9db17a8431ed9c, user_id=94bd5e48a65f4989bd0d69ef7351ef02, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.267918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.355079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1030/2098] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:19 2022] PUT /identity/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c/users/94bd5e48a65f4989bd0d69ef7351ef02/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:19.359672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.373011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:19.380695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdda227-0090-4539-bdb0-661d880fc4f8 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.458335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.459020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.459332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.459605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.460359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.462034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa0bf-2373-47d5-b9ed-7bf861b4cd3d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.477062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1069/2099] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:19.481969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.500470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0037-e3f9-47e1-a4dc-3aeba9a95f5a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.508589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.510372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.510860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.511392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] PATH_INFO: `/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c/users/94bd5e48a65f4989bd0d69ef7351ef02/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.516147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e3b545c3fe4590ab9db17a8431ed9c, user_id=94bd5e48a65f4989bd0d69ef7351ef02, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.521877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.533228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:19.538674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1031/2100] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:19 2022] PUT /identity/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c/users/94bd5e48a65f4989bd0d69ef7351ef02/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:19.560159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da563-2afd-4b97-b855-110686612e19 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.562017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da563-2afd-4b97-b855-110686612e19 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.562564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20da563-2afd-4b97-b855-110686612e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.604195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a20da563-2afd-4b97-b855-110686612e19 None None] MFA Rules not processed for user `94bd5e48a65f4989bd0d69ef7351ef02`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:19.610991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a20da563-2afd-4b97-b855-110686612e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.719566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1070/2101] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:19.733628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.738167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.831671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe84ff0-a843-4f66-9ba9-a75365e06a5f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.836590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.837263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.837664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.837958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.919318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.921492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db051c-0e8f-4aad-aad7-6e226093df75 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.932244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1032/2102] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:29.715093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=61 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:29.927720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=48 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:31.952389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20da563-2afd-4b97-b855-110686612e19 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.098157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20da563-2afd-4b97-b855-110686612e19 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.107664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.108930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.109288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.109640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] PATH_INFO: `/v3/users/5512aa89960645dabcdaba4d95d7cfa4` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.111258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5512aa89960645dabcdaba4d95d7cfa4)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.113565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:32.434003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5512aa89960645dabcdaba4d95d7cfa4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.453964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5512aa89960645dabcdaba4d95d7cfa4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.501106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5512aa89960645dabcdaba4d95d7cfa4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.502518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5512aa89960645dabcdaba4d95d7cfa4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.625212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5512aa89960645dabcdaba4d95d7cfa4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.632595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1071/2103] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:31 2022] DELETE /identity/v3/users/5512aa89960645dabcdaba4d95d7cfa4 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:32.639502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.640212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.640383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.666319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:32.667629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:32.828879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1033/2104] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:32.837526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.840089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:32.973280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d050d7-1edd-4caf-8c4e-7b959d1812cd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.978479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.979449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.979908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.980435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.992651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.994241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.013272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1072/2105] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:33.667287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.667287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.683029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bb334-c1c7-44b3-a8ec-1e028be63e0c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.688523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.689419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.689826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.689949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] PATH_INFO: `/v3/projects/f8770f840b8c45328b96b76b6f09e9fe` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.692694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8770f840b8c45328b96b76b6f09e9fe)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.695262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.908967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8770f840b8c45328b96b76b6f09e9fe', 'request_id': 'req-18786096-0ec3-4c32-9400-5734811febf9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.915347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8770f840b8c45328b96b76b6f09e9fe', 'request_id': 'req-18786096-0ec3-4c32-9400-5734811febf9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.925832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8770f840b8c45328b96b76b6f09e9fe', 'request_id': 'req-18786096-0ec3-4c32-9400-5734811febf9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.066164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] The token cache is being invalidate because project f8770f840b8c45328b96b76b6f09e9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:34.066552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.067323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.069331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1034/2106] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:33 2022] DELETE /identity/v3/projects/f8770f840b8c45328b96b76b6f09e9fe => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:34.074178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.086694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb8575-b9ff-4ad9-8a85-518a2e4bb8ed service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.109561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:34.162401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.163479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.163812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.164171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] PATH_INFO: `/v3/users/94bd5e48a65f4989bd0d69ef7351ef02` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.165638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bd5e48a65f4989bd0d69ef7351ef02)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.167819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.588057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bd5e48a65f4989bd0d69ef7351ef02'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.605436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bd5e48a65f4989bd0d69ef7351ef02'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.631330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bd5e48a65f4989bd0d69ef7351ef02'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.632519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bd5e48a65f4989bd0d69ef7351ef02'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.836185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bd5e48a65f4989bd0d69ef7351ef02'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.843413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1073/2107] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:34 2022] DELETE /identity/v3/users/94bd5e48a65f4989bd0d69ef7351ef02 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:35.602375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:35.603993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.620246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786096-0ec3-4c32-9400-5734811febf9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.711346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.712648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.713117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.713525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] PATH_INFO: `/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.715968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e3b545c3fe4590ab9db17a8431ed9c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.719998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.757617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:35.759221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.774959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10569d5-ca44-442a-9bed-8a17d0ca7c41 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.784865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.786018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.786636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.787060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] PATH_INFO: `/v3/users/ea3b131d8dfb442e8076c73989442490` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.793196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3b131d8dfb442e8076c73989442490)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.796202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.879684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:35.913382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e3b545c3fe4590ab9db17a8431ed9c', 'request_id': 'req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.921877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e3b545c3fe4590ab9db17a8431ed9c', 'request_id': 'req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.932506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e3b545c3fe4590ab9db17a8431ed9c', 'request_id': 'req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.058283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] The token cache is being invalidate because project b4e3b545c3fe4590ab9db17a8431ed9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:36.058283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.060986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-30fc0a1b-a208-4634-b472-8e7ccdfe94b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.064344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1035/2108] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:35 2022] DELETE /identity/v3/projects/b4e3b545c3fe4590ab9db17a8431ed9c => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:36.072516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2855-2ecc-4046-8924-ba8d343b1ce9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.072974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2855-2ecc-4046-8924-ba8d343b1ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.073402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2855-2ecc-4046-8924-ba8d343b1ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.100535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-57ba2855-2ecc-4046-8924-ba8d343b1ce9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:36.101968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-57ba2855-2ecc-4046-8924-ba8d343b1ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.135514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3b131d8dfb442e8076c73989442490'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.148789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3b131d8dfb442e8076c73989442490'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.165022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3b131d8dfb442e8076c73989442490'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.165891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3b131d8dfb442e8076c73989442490'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.194521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1036/2109] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:36.206144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.206431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.206712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.228277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:36.230207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.244970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1037/2110] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:36.253787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-40b0b43e-ac3f-47f8-85f5-42f9e8ebbed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3b131d8dfb442e8076c73989442490'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:36.254203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.264888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1074/2111] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:35 2022] DELETE /identity/v3/users/ea3b131d8dfb442e8076c73989442490 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:36.272425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595c71-e32a-4798-9c96-cdf4f117eba6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.272630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.272630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.272630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.293044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:36.295390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.306069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:36.360638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.361632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.362056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.362429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.363385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:36.363854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.365956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.385245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1038/2112] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:36.387515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.393155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1075/2113] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:36.407342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f096b9-8d08-4675-9aaf-4415cbcfb17d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.408937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.411369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.413060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.413735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.414019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.414293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.415001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.416981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.498252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2678dc-32e0-4a45-bef8-8fa1c1dc8db1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.501827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.502525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.502785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.503062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.514888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.516379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.526832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1076/2114] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:36.757245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1039/2115] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/projects => generated 371 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:36.763951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.781456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55720-8082-4977-9650-4da7abf4e30b service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.872538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.873155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.873509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.873853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.874558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.876563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.087489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1077/2116] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/users => generated 286 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:37.092632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.106499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0377-b55d-43bb-85f8-0f53ba38aaab admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.112294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.113384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.113833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.114263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.115234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.117137 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.132075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1040/2117] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:37.136748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.162061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea68fe-8639-4b12-a548-66031dccdb3c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.201307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.202345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.223510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.223510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.223510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.223510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.237871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1078/2118] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:37.242151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.256437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa137-c56a-4405-84d3-59efae9caa5c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.275291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.275291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.275291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.275291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] PATH_INFO: `/v3/projects/a6387bd82b5f489397a560bdf4da437b/users/bd56f1d650194b73a74c01446fa7f23e/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.276742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:37.293321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6387bd82b5f489397a560bdf4da437b, user_id=bd56f1d650194b73a74c01446fa7f23e, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.296288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c75c47-c902-41fd-b231-0ef68564ae04 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.494246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.495624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1041/2119] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:37 2022] PUT /identity/v3/projects/a6387bd82b5f489397a560bdf4da437b/users/bd56f1d650194b73a74c01446fa7f23e/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:37.497979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:37.508938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.509234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.509444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.527633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335236ff-b3ae-4f52-aa46-45ecba026dde admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.602541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] MFA Rules not processed for user `bd56f1d650194b73a74c01446fa7f23e`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:37.602541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:37.602995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:37.645446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.646860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.647381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.647858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] PATH_INFO: `/v3/projects/4b73048e43674902b47bf7e2f38388cf` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.652044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b73048e43674902b47bf7e2f38388cf)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.656808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.700336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1042/2120] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:37.886093 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b73048e43674902b47bf7e2f38388cf', 'request_id': 'req-3d1d07a1-f6d4-4bca-a595-d168d868427d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:37.892894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b73048e43674902b47bf7e2f38388cf', 'request_id': 'req-3d1d07a1-f6d4-4bca-a595-d168d868427d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.042395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b73048e43674902b47bf7e2f38388cf', 'request_id': 'req-3d1d07a1-f6d4-4bca-a595-d168d868427d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.056415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] The token cache is being invalidate because project 4b73048e43674902b47bf7e2f38388cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:38.056711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.057388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.060331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1079/2121] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:37 2022] DELETE /identity/v3/projects/4b73048e43674902b47bf7e2f38388cf => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:38.064547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.076517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fce4b-5cef-40ee-a126-56ec1ccd9251 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.111557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.118838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.171727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.172654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.172944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.173247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] PATH_INFO: `/v3/users/4cfaf7f2446542b0a1c520771d2fc125` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.189151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfaf7f2446542b0a1c520771d2fc125)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.189151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.206530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d07a1-f6d4-4bca-a595-d168d868427d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.217595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.218722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.219244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.219723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.221979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.291980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.294356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.304239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1080/2122] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:38.331602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.333324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.349044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cb5a9-a083-49dd-a3f8-36eba33cf9e3 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.354968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.356727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.357039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.357402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] PATH_INFO: `/v3/users/bd56f1d650194b73a74c01446fa7f23e` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.360222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd56f1d650194b73a74c01446fa7f23e)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.362136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.533946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfaf7f2446542b0a1c520771d2fc125'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.535016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfaf7f2446542b0a1c520771d2fc125'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.557806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfaf7f2446542b0a1c520771d2fc125'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.571343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfaf7f2446542b0a1c520771d2fc125'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.786211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-bc06fb77-eeb1-4fe6-9b7a-6a0248bd668d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfaf7f2446542b0a1c520771d2fc125'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.790623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:38.795859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1043/2123] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:38 2022] DELETE /identity/v3/users/4cfaf7f2446542b0a1c520771d2fc125 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:38.812022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd56f1d650194b73a74c01446fa7f23e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.825903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd56f1d650194b73a74c01446fa7f23e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.842930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd56f1d650194b73a74c01446fa7f23e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.843935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd56f1d650194b73a74c01446fa7f23e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.076326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd56f1d650194b73a74c01446fa7f23e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.083787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1081/2124] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:38 2022] DELETE /identity/v3/users/bd56f1d650194b73a74c01446fa7f23e => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:39.101916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.101916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.101916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.109171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:39.110625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.208742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1044/2125] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:39.220379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.222269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.369340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611c76c-0929-4376-b52d-fdfc2dc64094 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.369340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:39.369969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.369969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.370824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.383039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:39.384546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:39.395246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1082/2126] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:39.455923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:39.945345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.947577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.962611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153117f9-3593-4e8b-b0b3-197402232cec None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.979763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:39.981311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.981602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.981884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] PATH_INFO: `/v3/projects/34203791a0d64f23b88cb3f4e47b52f8` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.983539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34203791a0d64f23b88cb3f4e47b52f8)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:39.991953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.208062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34203791a0d64f23b88cb3f4e47b52f8', 'request_id': 'req-314b41d0-b82a-4521-9288-454dddeeaf6a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.217829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34203791a0d64f23b88cb3f4e47b52f8', 'request_id': 'req-314b41d0-b82a-4521-9288-454dddeeaf6a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.227409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34203791a0d64f23b88cb3f4e47b52f8', 'request_id': 'req-314b41d0-b82a-4521-9288-454dddeeaf6a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.420374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] The token cache is being invalidate because project 34203791a0d64f23b88cb3f4e47b52f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:40.420374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.420374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.420374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1045/2127] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:39 2022] DELETE /identity/v3/projects/34203791a0d64f23b88cb3f4e47b52f8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:40.423017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.431368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.438849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.441900 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ef3a-9310-48d0-a590-886c172cbf9b service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.587875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.589005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.589456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.589745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] PATH_INFO: `/v3/users/b42bed9d9d2e4725976a3f0cb6dd24f9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.591361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42bed9d9d2e4725976a3f0cb6dd24f9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.593343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.594402 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b41d0-b82a-4521-9288-454dddeeaf6a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.599486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.600422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.600702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.600978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] PATH_INFO: `/v3/projects/a6387bd82b5f489397a560bdf4da437b` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.602260 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6387bd82b5f489397a560bdf4da437b)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.605928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.903136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42bed9d9d2e4725976a3f0cb6dd24f9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.907275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6387bd82b5f489397a560bdf4da437b', 'request_id': 'req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.916835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6387bd82b5f489397a560bdf4da437b', 'request_id': 'req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.917033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42bed9d9d2e4725976a3f0cb6dd24f9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.930882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6387bd82b5f489397a560bdf4da437b', 'request_id': 'req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.940061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42bed9d9d2e4725976a3f0cb6dd24f9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.940886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42bed9d9d2e4725976a3f0cb6dd24f9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.014901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] The token cache is being invalidate because project a6387bd82b5f489397a560bdf4da437b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:41.015237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.015916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d3f37406-dc02-4251-9f6e-d9fcfa7fd597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.017627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1046/2128] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:40 2022] DELETE /identity/v3/projects/a6387bd82b5f489397a560bdf4da437b => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:41.022473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.022766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.023180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.046975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.049744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.125372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b7d71e2f-f83a-43d4-94f6-d269eb93130b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42bed9d9d2e4725976a3f0cb6dd24f9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.143449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1083/2129] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:40 2022] DELETE /identity/v3/users/b42bed9d9d2e4725976a3f0cb6dd24f9 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:41.186280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1047/2130] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:41.188488 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.188985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.189334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.227236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.228952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.250540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:41.337658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1084/2131] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:41.342657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.354464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21ed3-9488-4525-8eb0-91ed35eb4a86 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.362652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.363540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.363981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.364268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.365027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:41.365451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.367223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.387718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1048/2132] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:41.389833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.400676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f769-f0d3-42dc-849f-b60ea3d3d946 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.412289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.413556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.413954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.414249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.415282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.417608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.581995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1085/2133] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:41.588607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.607176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:41.615782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ef555-10a4-43cd-88ed-9552c585d65f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.705153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.706145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.706529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.706909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.707747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.709818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.895920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1049/2134] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:41.899896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.917314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803ead9-50cb-46e7-badb-d10a67b61de7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.932639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.932639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.932639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.932639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.933041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.933041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.934293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.938722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.952383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1086/2135] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:41.953863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.962985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64df846-0440-4ed2-9daa-b13519e6deba admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.966697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea3b9-8aba-4b08-8dc3-1927b0b556b9 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.970495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.971157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.971157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.971741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] PATH_INFO: `/v3/projects/cd1589d915834f30878806fdf35cd981` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.972834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1589d915834f30878806fdf35cd981)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.973326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.974153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.974684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.974786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.975071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.975926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.977639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.991246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1087/2136] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:41.995611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.009476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690b04-e5ad-4870-87dc-8def6e0b3f1e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.015367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.016286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.016568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.016847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] PATH_INFO: `/v3/projects/20b87d0f8d1c4ab8a9218ec96872750f/users/41bc56e0e3d0451482572959451ae043/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.032955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b87d0f8d1c4ab8a9218ec96872750f, user_id=41bc56e0e3d0451482572959451ae043, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.035583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9f7c-4ea1-4c08-8853-f2e1d57058e4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.139984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1088/2137] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:41 2022] PUT /identity/v3/projects/20b87d0f8d1c4ab8a9218ec96872750f/users/41bc56e0e3d0451482572959451ae043/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:42.145175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.145765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.146649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.158456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1589d915834f30878806fdf35cd981', 'request_id': 'req-f29d0a64-69db-4166-b86d-ce3316a2b56f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:42.169233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:42.169612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1589d915834f30878806fdf35cd981', 'request_id': 'req-f29d0a64-69db-4166-b86d-ce3316a2b56f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:42.170628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.177321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1589d915834f30878806fdf35cd981', 'request_id': 'req-f29d0a64-69db-4166-b86d-ce3316a2b56f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:42.244446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1089/2138] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:42.264847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.268203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.299151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] The token cache is being invalidate because project cd1589d915834f30878806fdf35cd981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:42.299628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:42.300348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f29d0a64-69db-4166-b86d-ce3316a2b56f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:42.307636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1050/2139] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:41 2022] DELETE /identity/v3/projects/cd1589d915834f30878806fdf35cd981 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:42.313387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafc7f-1d77-445a-8ade-ed7a85cb368d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.313884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafc7f-1d77-445a-8ade-ed7a85cb368d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.314216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbafc7f-1d77-445a-8ade-ed7a85cb368d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.346186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fdbafc7f-1d77-445a-8ade-ed7a85cb368d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:42.348127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fdbafc7f-1d77-445a-8ade-ed7a85cb368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.361208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1051/2140] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:42.369557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.369853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.370153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.374162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a269-1084-4ddd-a64a-6e89d8648d3e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.378797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.379573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.386856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.386856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.386856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.396914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.399406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.400487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:42.403322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.415245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1052/2141] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:42.419207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.428603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1090/2142] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:42.436463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f19ecf-3e98-4165-ab58-cfd37b4c19a2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.446188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.449840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.449840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.449840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.449840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:42.449840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.453032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.474016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1053/2143] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:42.478884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.490591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781436c-63fa-45aa-bd58-74013c4e8910 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.497521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.498012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.498209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.500910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.500910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.501500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.609743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1091/2144] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:42.610861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.622913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be741b-965d-4b23-ad2b-e1cfc11f3a64 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.726066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.727305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.727837 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.728474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.729839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.734692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.896101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1054/2145] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:42.900290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.918724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b65e0-70ed-4711-aa93-a0f5d2765254 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.923924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.924707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.926092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.926092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.927968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.928141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.946406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1092/2146] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:42.953254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.970433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe377e7-9fcc-4cf5-a6be-23134410a898 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.977926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.978615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.979108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.979637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.981075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.984295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d904ec1-ef6c-44a3-a7e8-52b90e5dc625 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.998819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1055/2147] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:43.009717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:43.024125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202529e9-fc35-4c9e-bdd7-dd346d67421f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:43.033296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:43.034781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:43.035063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:43.035662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] PATH_INFO: `/v3/projects/2eb05bae9b0b403c8834967ed1d689ab/users/47a4727579ef4c439e6b0d5c56ebef01/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:43.068640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb05bae9b0b403c8834967ed1d689ab, user_id=47a4727579ef4c439e6b0d5c56ebef01, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:43.071656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:43.200636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1093/2148] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:43 2022] PUT /identity/v3/projects/2eb05bae9b0b403c8834967ed1d689ab/users/47a4727579ef4c439e6b0d5c56ebef01/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:43.214478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:43.214478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:43.218841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:43.248264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] MFA Rules not processed for user `47a4727579ef4c439e6b0d5c56ebef01`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:43.249840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:43.271044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:43.337793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1056/2149] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:43.350541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:43.356017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:43.455102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d885ac1-075d-462f-b18d-450bd4ad785c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:43.459419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:43.460236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:43.460615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:43.461303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:43.474297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:43.476769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:43.492929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1094/2150] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:43.975387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:44.500003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:44.502802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:44.665517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56befb9-318d-47ca-bf83-6ca488f569bf None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:44.672965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:44.674019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:44.674393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:44.674807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] PATH_INFO: `/v3/users/2c8ce3b0a82f47dc9ae6a4fb2abcaeaa` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:44.676702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8ce3b0a82f47dc9ae6a4fb2abcaeaa)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:44.678989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:44.952192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8ce3b0a82f47dc9ae6a4fb2abcaeaa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:44.952192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8ce3b0a82f47dc9ae6a4fb2abcaeaa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:44.966255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8ce3b0a82f47dc9ae6a4fb2abcaeaa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:44.974118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8ce3b0a82f47dc9ae6a4fb2abcaeaa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.093533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f53846ae-3b96-4586-85b2-fa63ad55fde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8ce3b0a82f47dc9ae6a4fb2abcaeaa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.102412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1057/2151] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:44 2022] DELETE /identity/v3/users/2c8ce3b0a82f47dc9ae6a4fb2abcaeaa => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:45.410442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:45.430729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea174b-f383-4da5-bb9f-5b1001255241 service glance] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:45.525219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:45.528458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.528955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.529413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] PATH_INFO: `/v3/projects/457a247df22a45b39318dd2add051120` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.531078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=457a247df22a45b39318dd2add051120)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:45.533470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:45.723049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457a247df22a45b39318dd2add051120', 'request_id': 'req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.733142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457a247df22a45b39318dd2add051120', 'request_id': 'req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.736863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:45.840870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457a247df22a45b39318dd2add051120', 'request_id': 'req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.854567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] The token cache is being invalidate because project 457a247df22a45b39318dd2add051120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:45.854915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.855675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9b3a4dd6-d6f3-4d97-a5d1-acbcea9c9fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.857340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1095/2152] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:45 2022] DELETE /identity/v3/projects/457a247df22a45b39318dd2add051120 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:45.862632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21416f0-c007-4191-862b-547d33580d0a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.862932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21416f0-c007-4191-862b-547d33580d0a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.863293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21416f0-c007-4191-862b-547d33580d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.896088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a21416f0-c007-4191-862b-547d33580d0a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:45.897989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a21416f0-c007-4191-862b-547d33580d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:45.985216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1058/2153] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:45.989538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.990450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.991071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.017373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:46.019561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.034281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1096/2154] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:46.044839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21416f0-c007-4191-862b-547d33580d0a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.055383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21416f0-c007-4191-862b-547d33580d0a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.061620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.062321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.062865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.062865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.063661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:46.063975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.065628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.078382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1059/2155] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:46.082904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.092679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa368954-2380-438d-826c-7669c9f0ca0b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.097962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.099865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.099865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.099865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.100283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.101939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.214735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:46.261198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1097/2156] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:46.265575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.277538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ebad7-79b4-4b44-a8bd-799e77a96e1e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.374689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.375430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.375885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.376398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.377259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.379263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.476814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1060/2157] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:46.478024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.491891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b9a93-9017-441b-b983-2bb84d264385 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.505156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.506203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.506853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.512365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.512365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.512365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.530473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1098/2158] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:46.531688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.550480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e10d1-fedc-48dd-addd-46b3f8b1d618 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.556628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.563647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.563647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.563647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.563647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.563647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba722b-3eb9-45cd-91f0-472d121508f0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.580761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1061/2159] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:46.581848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.596119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb328a-146b-4f72-8bf0-fdd0bfc21753 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.604362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.607180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.607690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.611720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] PATH_INFO: `/v3/projects/6cf0cfc65d914753b1e19db9e238b4dd/users/4ebc96dc6b16411a9adb8e91793cda07/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.629087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf0cfc65d914753b1e19db9e238b4dd, user_id=4ebc96dc6b16411a9adb8e91793cda07, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.634767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.739506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1099/2160] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:46 2022] PUT /identity/v3/projects/6cf0cfc65d914753b1e19db9e238b4dd/users/4ebc96dc6b16411a9adb8e91793cda07/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:46.746469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.747542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.747542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.769871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:46.771114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.864480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1062/2161] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:46.885325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.889361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.970605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5687-8495-48f0-83e4-d65caa8711a7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.975539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.976521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.976804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.977106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.987856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.989387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb9cfb-b4ba-4846-ac99-297816b42bc4 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:47.001010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1100/2162] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:48.882277 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:48.984111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:48.986087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:49.001625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39225a-a123-42d7-94d8-0893b00fc20a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:49.017008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:49.018027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.018547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.018901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] PATH_INFO: `/v3/users/34ad740efe5c45dd9b821b3aa6482a6b` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.020973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ad740efe5c45dd9b821b3aa6482a6b)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:49.025164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:49.463059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ad740efe5c45dd9b821b3aa6482a6b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.473209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ad740efe5c45dd9b821b3aa6482a6b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.508074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ad740efe5c45dd9b821b3aa6482a6b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.528590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ad740efe5c45dd9b821b3aa6482a6b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.858023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ad740efe5c45dd9b821b3aa6482a6b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.869238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1063/2163] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:48 2022] DELETE /identity/v3/users/34ad740efe5c45dd9b821b3aa6482a6b => generated 0 bytes in 887 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:49.872516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524ac5-2ce9-4f11-9a3a-1094a61dfcef None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.873006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524ac5-2ce9-4f11-9a3a-1094a61dfcef None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.873392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524ac5-2ce9-4f11-9a3a-1094a61dfcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.892635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:49.926062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d5524ac5-2ce9-4f11-9a3a-1094a61dfcef None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:49.930558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5524ac5-2ce9-4f11-9a3a-1094a61dfcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:50.159922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1101/2164] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:50.172339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:50.174592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:50.302378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906a56-b49c-439b-85e0-59e61a96aa32 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:50.314625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.315739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.316234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.316731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.356234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.366874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.377718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1064/2165] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:50.818823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.819347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.821061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.890523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] MFA Rules not processed for user `41bc56e0e3d0451482572959451ae043`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:50.893056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:51.052477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1102/2166] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:51.052722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.052722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:51.052998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d037f-61d4-4a54-a8b6-4faf3bef1387 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.059230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.060292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.060971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.061529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.072267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.077236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.296603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.297315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1065/2167] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:51 2022] POST /identity/v3/projects => generated 383 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:51.326498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecd092-00b1-402f-9199-76825a5c65b1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.452123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.452898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.453258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.453612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.454388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.456470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.481487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.484203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:51.501221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7997af-e9ab-4272-9caf-b47b9528a177 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.514797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.516336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.516981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.517430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] PATH_INFO: `/v3/projects/4f4c143f42cf4babaca91135c6fe7817` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.519797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4c143f42cf4babaca91135c6fe7817)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.523275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.569929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1103/2168] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:51 2022] POST /identity/v3/users => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:51.571322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.585837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec918b8-45a8-467e-b337-707661d92e17 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.592305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.593246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.593678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.594124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.595035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.596880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.611513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1104/2169] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:51.615076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.629672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b2995-4f19-4425-a718-5bfc2c629389 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.634898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.648063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.648063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.648063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] PATH_INFO: `/v3/projects/c5fabcb3cf23446192558faa11c75e60/users/99463be973b5492297d1a291323a77be/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.661271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fabcb3cf23446192558faa11c75e60, user_id=99463be973b5492297d1a291323a77be, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.664256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.817578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1105/2170] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:51 2022] PUT /identity/v3/projects/c5fabcb3cf23446192558faa11c75e60/users/99463be973b5492297d1a291323a77be/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:51.817967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4c143f42cf4babaca91135c6fe7817', 'request_id': 'req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:51.819738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:51.837475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4c143f42cf4babaca91135c6fe7817', 'request_id': 'req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:51.847369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4c143f42cf4babaca91135c6fe7817', 'request_id': 'req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:51.914708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bfaad-b5bd-4422-a45b-f711705f0252 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:51.921499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:51.922223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.922506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.922781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.923714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:51.925854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:51.934070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] The token cache is being invalidate because project 4f4c143f42cf4babaca91135c6fe7817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:51.938451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:51.938451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-05ef81f2-a2c0-4e79-b50f-a306270bd0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:51.939057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1066/2171] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:51 2022] DELETE /identity/v3/projects/4f4c143f42cf4babaca91135c6fe7817 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:51.944548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d829-eecd-4beb-9770-0ea8c5f340a7 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:51.944981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d829-eecd-4beb-9770-0ea8c5f340a7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:51.945449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d829-eecd-4beb-9770-0ea8c5f340a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:51.988979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1106/2172] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:52.002451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:52.004745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:52.012353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9484d829-eecd-4beb-9770-0ea8c5f340a7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:52.014618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9484d829-eecd-4beb-9770-0ea8c5f340a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:52.028021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493c93-6fa9-40bf-92a0-7eaaf617b030 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:52.039781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1067/2173] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:52.048825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:52.049071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:52.049434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:52.049788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:52.050083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:52.050393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:52.050830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] PATH_INFO: `/v3/projects/c5fabcb3cf23446192558faa11c75e60/users/99463be973b5492297d1a291323a77be/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:52.053173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fabcb3cf23446192558faa11c75e60, user_id=99463be973b5492297d1a291323a77be, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:52.056515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:52.075792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1107/2174] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:52 2022] PUT /identity/v3/projects/c5fabcb3cf23446192558faa11c75e60/users/99463be973b5492297d1a291323a77be/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:52.096412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:52.098062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:52.291160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1068/2175] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:52.297749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:52.314440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:52.333918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b667-7508-451d-8d59-26c23b636447 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:52.353749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:52.354921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:52.355435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:52.358246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:52.358246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:52.358246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:52.363479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:52.411973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:52.414757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1108/2176] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:52.427951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834c67-acda-46a4-bb99-3cd2df3b4cb0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:52.444294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:52.447682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:52.449125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:52.449653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:52.454477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:52.463692 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:52.772442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1069/2177] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:52 2022] POST /identity/v3/projects => generated 369 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:52.780455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:52.799686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97d7c4-003d-487c-8fdf-2e0dc30da993 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:52.935835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:52.937357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:52.937870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:52.938610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:52.939915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:52.942431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:52.972327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:52.974309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:52.997064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743de954-6a6e-40ec-ad56-3cd77583144e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.016376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.019880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.021495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.022595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] PATH_INFO: `/v3/users/cb54f74f2fd74f09b6424bc6c81f04a1` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.028597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb54f74f2fd74f09b6424bc6c81f04a1)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.031528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.085430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1109/2178] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:52 2022] POST /identity/v3/users => generated 285 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:53.086449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.123903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcba54-68e8-433c-ab54-2dc42925bc8e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.136169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.137343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.138432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.138992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.140318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.150167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.179741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1110/2179] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:53.189302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.225926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0b880-348c-4d45-bb13-08de1765b306 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.239191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.241412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.242208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.243255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.247179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.256510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.287495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1111/2180] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:53.293285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.312834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb54f74f2fd74f09b6424bc6c81f04a1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:53.314312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb54f74f2fd74f09b6424bc6c81f04a1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:53.318201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17375c7-7f9a-411d-b9fa-222177721076 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.325618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.326934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.327264 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.329006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] PATH_INFO: `/v3/projects/b579dcf22a78460eba58bc42e539f8c6/users/48d795546ed24bf784051480712ef2d3/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.346934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb54f74f2fd74f09b6424bc6c81f04a1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:53.360232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b579dcf22a78460eba58bc42e539f8c6, user_id=48d795546ed24bf784051480712ef2d3, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.365274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dbd49-7ed2-4bc7-b4eb-b8902b1deca6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.376195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb54f74f2fd74f09b6424bc6c81f04a1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:53.414281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:53.511154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1112/2181] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:53 2022] PUT /identity/v3/projects/b579dcf22a78460eba58bc42e539f8c6/users/48d795546ed24bf784051480712ef2d3/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:53.520810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.520810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.521165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb54f74f2fd74f09b6424bc6c81f04a1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:53.521251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.539576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1070/2182] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:52 2022] DELETE /identity/v3/users/cb54f74f2fd74f09b6424bc6c81f04a1 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:53.667883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] MFA Rules not processed for user `48d795546ed24bf784051480712ef2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:53.675241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:53.950616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1113/2183] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:53.962156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.966406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:54.082490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d050d-2c0e-48b8-9256-2da80c8ea439 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:54.087620 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:54.089222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:54.089983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:54.090611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:54.115674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:54.118078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:54.130562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1071/2184] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:54.217390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:54.221156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:54.479686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f582f-8c5e-4d38-b527-cf572bf974be None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:54.493229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:54.495626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:54.496136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:54.496647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] PATH_INFO: `/v3/projects/a2431cf8c26a4c9a8dffd0dc62831631` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:54.499571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2431cf8c26a4c9a8dffd0dc62831631)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:54.504235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:54.808718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2431cf8c26a4c9a8dffd0dc62831631', 'request_id': 'req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:54.825079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2431cf8c26a4c9a8dffd0dc62831631', 'request_id': 'req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:54.937868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2431cf8c26a4c9a8dffd0dc62831631', 'request_id': 'req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:54.961219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] The token cache is being invalidate because project a2431cf8c26a4c9a8dffd0dc62831631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:54.961912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:54.963091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:54.968200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1114/2185] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:54 2022] DELETE /identity/v3/projects/a2431cf8c26a4c9a8dffd0dc62831631 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:54.972468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:54.986956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348037-5dba-4412-b41c-82abd3de0000 service glance] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:55.159985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:55.160957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:55.161306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:55.161585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] PATH_INFO: `/v3/users/31c6117ef2f343e8b7b329e5dede13d0` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:55.163702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c6117ef2f343e8b7b329e5dede13d0)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:55.166091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:55.552535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c6117ef2f343e8b7b329e5dede13d0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.553511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c6117ef2f343e8b7b329e5dede13d0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.570607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c6117ef2f343e8b7b329e5dede13d0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.579597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c6117ef2f343e8b7b329e5dede13d0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.603235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:55.695432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c6117ef2f343e8b7b329e5dede13d0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.702911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1072/2186] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:54 2022] DELETE /identity/v3/users/31c6117ef2f343e8b7b329e5dede13d0 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:55.795008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:56.266276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:56.268423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:56.297425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e5b68-d21f-4448-9a55-4e3a7ce151f0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:56.438681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:56.440250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:56.440752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:56.441368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] PATH_INFO: `/v3/projects/1e6ac66f61084132bb7f30b39245afbc` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:56.443389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6ac66f61084132bb7f30b39245afbc)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:56.445651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:56.686660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6ac66f61084132bb7f30b39245afbc', 'request_id': 'req-0b96b038-135e-49c9-be2d-24cc955bcbde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.700545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6ac66f61084132bb7f30b39245afbc', 'request_id': 'req-0b96b038-135e-49c9-be2d-24cc955bcbde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.808099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:56.811844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6ac66f61084132bb7f30b39245afbc', 'request_id': 'req-0b96b038-135e-49c9-be2d-24cc955bcbde'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.812224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:56.841143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] The token cache is being invalidate because project 1e6ac66f61084132bb7f30b39245afbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:56.842294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.849538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0b96b038-135e-49c9-be2d-24cc955bcbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.856249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1115/2187] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:56 2022] DELETE /identity/v3/projects/1e6ac66f61084132bb7f30b39245afbc => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:56.862543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761fa0-2d72-40e1-98cd-ef381e7ab077 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:56.863150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761fa0-2d72-40e1-98cd-ef381e7ab077 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:56.864207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761fa0-2d72-40e1-98cd-ef381e7ab077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:56.869694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9260e0-a2ca-4c07-8458-06d9644001a0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:56.928406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ed761fa0-2d72-40e1-98cd-ef381e7ab077 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:56.931459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ed761fa0-2d72-40e1-98cd-ef381e7ab077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.084251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.085675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.086339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.086339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] PATH_INFO: `/v3/users/1e326eaff59445958ceddac5db96bd19` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.090730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e326eaff59445958ceddac5db96bd19)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.106954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.159705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1116/2188] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:57.172381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.172773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.173062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.210431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:57.211792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.222726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1117/2189] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:57.227733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.240117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9d2e-6e5a-48b8-b37e-32bfcd4fe0a9 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.246728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.247893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.248441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.248922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.249801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:57.250311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.252064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.267294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1118/2190] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:57.277083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.293899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b0bd-cc45-4e4a-88e8-0d2c65de5655 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.299394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.300303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.300591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.300877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.301697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.305064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.449817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e326eaff59445958ceddac5db96bd19'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.453334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e326eaff59445958ceddac5db96bd19'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.468331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1119/2191] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:57.473219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.475221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.483315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e326eaff59445958ceddac5db96bd19'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.495615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e326eaff59445958ceddac5db96bd19'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.569250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c20cf3-e609-4fa2-9c22-a25b17d45011 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.576467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.577373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.577674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.577958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.582250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.582250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.629100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2406f40e-bc29-497c-a362-d5f30f97d66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e326eaff59445958ceddac5db96bd19'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.636079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1073/2192] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:56 2022] DELETE /identity/v3/users/1e326eaff59445958ceddac5db96bd19 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:57.641774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.642060 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.647862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.679461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:57.683736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.706051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1120/2193] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:57.707056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.718596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71da079-72f6-45b0-9b03-73960df34232 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.821949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1074/2194] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:57.896479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.898869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.940144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:57.969455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.971305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.974995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.975563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.979108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.981107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e60878-bd86-4094-b33b-f941590ce5df None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.983216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.985356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.986375 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.986813 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.988391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.001514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.013246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.016873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1075/2195] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:58.030710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1121/2196] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:58.032087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.060664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b068f2-b56e-499c-b9dc-f2705068a305 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.095994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.096710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.097001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.097283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.098608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.101128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2cc6a-03b1-4e6e-8b48-a493b695b789 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.116124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1076/2197] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:58.137160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.154352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f0648-590e-424a-8a7e-8ebc946e61a0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.166910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.169708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.170809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.171225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] PATH_INFO: `/v3/projects/04ddf167bad141ce9a9651bcb89d4e7f/users/19d8200cf5724bd9b647263da7a804bf/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.220445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ddf167bad141ce9a9651bcb89d4e7f, user_id=19d8200cf5724bd9b647263da7a804bf, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.224618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.389353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1122/2198] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:58 2022] PUT /identity/v3/projects/04ddf167bad141ce9a9651bcb89d4e7f/users/19d8200cf5724bd9b647263da7a804bf/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:58.391861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.392351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.392826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.552628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] MFA Rules not processed for user `19d8200cf5724bd9b647263da7a804bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:58.555971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:58.716788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1077/2199] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:58.721215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.881755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5e7a9-cd2c-4fa0-8b1c-96b25b6b6ff0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.892761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.893567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.894294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.894750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.895943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.898349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd480f-55f1-4e70-9f07-48dbf2ddb9c6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.066392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1123/2200] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/projects => generated 383 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:59.067423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.091095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42ad5-d033-4b2f-877f-c3e840289b28 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.138740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.139166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.139435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.203204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] MFA Rules not processed for user `4ebc96dc6b16411a9adb8e91793cda07`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:59.208554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:59.302879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:59.307467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.308808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.309219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.310109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.310932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.316704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.393159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1124/2201] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:59.401676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.403740 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:59.485966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1078/2202] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:59 2022] POST /identity/v3/users => generated 298 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:59.492400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.543399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8ba9-8484-4580-8e98-6a37dcb14417 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.576910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.577904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.578076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.578227 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.587831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.587831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.607962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1079/2203] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:59.609677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.641838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fd8d9-6680-4182-b233-f94efc0d01d0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.647522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.649297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.649589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.649916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] PATH_INFO: `/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62/users/c39b9e278b904d3ea8188c3ae48bd7c2/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.670395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e63cf-b5c6-4bbb-a508-bf6af7ee752f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.680486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.680749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf5c1f65c2945299ad62ece75dfdd62, user_id=c39b9e278b904d3ea8188c3ae48bd7c2, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.682643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.683102 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.683611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.684130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.698893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.702332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.723006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1125/2204] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:59.881379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1080/2205] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:59 2022] PUT /identity/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62/users/c39b9e278b904d3ea8188c3ae48bd7c2/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:59.886127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.898254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dfd9d-dae9-4cb9-b00d-e42256356147 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.967697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.974265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.097857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:00.099262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.106545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.106545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.111353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:00.115704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:00.149332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1126/2206] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:00.156994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:00.191072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a005-8f13-42eb-a20c-1b7687426d99 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:00.201420 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:00.201629 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317414-6f3c-4b7f-9da1-9ba0b0251c41 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:00.203265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.203734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.204291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] PATH_INFO: `/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62/users/c39b9e278b904d3ea8188c3ae48bd7c2/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.207505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf5c1f65c2945299ad62ece75dfdd62, user_id=c39b9e278b904d3ea8188c3ae48bd7c2, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:00.211250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b019c-ef89-4b7c-bf06-5df2caaf0b16 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:00.215470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:00.216622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.217124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.217215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] PATH_INFO: `/v3/projects/976f72f4873540229a0ff788c89ebb8a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.220377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=976f72f4873540229a0ff788c89ebb8a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:00.222456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:00.239907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1127/2207] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:00 2022] PUT /identity/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62/users/c39b9e278b904d3ea8188c3ae48bd7c2/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:19:00.241861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.245724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.245724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.357083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] MFA Rules not processed for user `c39b9e278b904d3ea8188c3ae48bd7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:00.365791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.455538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '976f72f4873540229a0ff788c89ebb8a', 'request_id': 'req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:00.456551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '976f72f4873540229a0ff788c89ebb8a', 'request_id': 'req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:00.481962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '976f72f4873540229a0ff788c89ebb8a', 'request_id': 'req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:00.500009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1128/2208] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:00.515353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:00.518049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.634664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] The token cache is being invalidate because project 976f72f4873540229a0ff788c89ebb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:00.640832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:00.641320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8ed2d-f1ec-4c20-8166-3f0302534b4e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:00.644128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-48e4fc31-6697-4ef8-8624-a4fa81ae0bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:00.647969 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:00.658327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1081/2209] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:59 2022] DELETE /identity/v3/projects/976f72f4873540229a0ff788c89ebb8a => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:19:00.665648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569fb36-935f-4cab-ac8a-4bd71b93f510 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.666674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569fb36-935f-4cab-ac8a-4bd71b93f510 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.667336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569fb36-935f-4cab-ac8a-4bd71b93f510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.750183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a569fb36-935f-4cab-ac8a-4bd71b93f510 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:00.752595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a569fb36-935f-4cab-ac8a-4bd71b93f510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.821904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:00.823647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.823647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.828522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.835534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.989728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1082/2210] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:00.993534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.994192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.994772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.008990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.011493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.031416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:01.033462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.038253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1129/2211] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:01.049228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1083/2212] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:01.054524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.093194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78124-62dd-415c-b072-784104e2722d service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.106604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.107419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.107795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.108149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.109582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:19:01.109582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.111944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.169221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1130/2213] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:19:01.169820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.174516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9d2c5-29e7-4088-8467-469c8bd2b3a1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.188537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.189249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.189536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.192676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.193439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.195450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.380828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1084/2214] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:01 2022] POST /identity/v3/projects => generated 371 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:01.382536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.394512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fade-82c5-4191-942a-6c0d21ff75ca admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.550519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.551451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.551821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.552111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.552899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.554921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.759554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1131/2215] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:01 2022] POST /identity/v3/users => generated 286 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:01.765504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.795834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be914c-aafa-4d80-b66c-c26eb7742844 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.820433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.821591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.822112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.823017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.831018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.857282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.894753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:01.902603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1085/2216] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:01.906827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.930773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7af86-3598-4542-84f1-8febff6faf1f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.953592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.954335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.954623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.954910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.955708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.957374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096762-3896-4c84-928e-ea3243216450 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.986909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1132/2217] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:01.991751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.013826 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9c7f-a3d6-45b4-8d06-61316aaf16c5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:02.027622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:02.029660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.030435 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.031175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] PATH_INFO: `/v3/projects/76d33745da9045b989026dad3b5b7abb/users/162160bbb86e40c9bb957a213034e25f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.070976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d33745da9045b989026dad3b5b7abb, user_id=162160bbb86e40c9bb957a213034e25f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:02.075665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.221673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1086/2218] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:01 2022] PUT /identity/v3/projects/76d33745da9045b989026dad3b5b7abb/users/162160bbb86e40c9bb957a213034e25f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:02.230202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948780a-389c-4a03-a1eb-56294bb18449 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.232335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948780a-389c-4a03-a1eb-56294bb18449 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.233177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948780a-389c-4a03-a1eb-56294bb18449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.344873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c948780a-389c-4a03-a1eb-56294bb18449 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:02.347059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c948780a-389c-4a03-a1eb-56294bb18449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.563363 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1133/2219] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:02.662189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.664162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.856215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f7a6b-4d34-4aad-9f97-22a8239d8b99 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:02.873594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:02.874557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.875012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.875504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.928231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:02.933945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78c05-50e3-4918-969f-60d4de356c84 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.961256 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1087/2220] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:03.916693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:06.020136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:06.709219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6caa1e-ed92-4f2e-939c-91ee7eeb13b7 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:06.710892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6caa1e-ed92-4f2e-939c-91ee7eeb13b7 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:06.727561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6caa1e-ed92-4f2e-939c-91ee7eeb13b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:06.794342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8e6caa1e-ed92-4f2e-939c-91ee7eeb13b7 None None] MFA Rules not processed for user `4ebc96dc6b16411a9adb8e91793cda07`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:06.797763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8e6caa1e-ed92-4f2e-939c-91ee7eeb13b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:06.912348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1134/2221] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:07.092088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:07.233850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:07.234444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:07.234444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:07.292933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] MFA Rules not processed for user `99463be973b5492297d1a291323a77be`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:07.294189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.393065 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:07.393599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:07.397421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:07.452499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] MFA Rules not processed for user `162160bbb86e40c9bb957a213034e25f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:07.456257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.468237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1088/2222] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:07.484385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:07.490757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.602051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c3ef2-739c-40b7-b731-fdd27a2e1966 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:07.609808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:07.610683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:07.610982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:07.611288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:07.620586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1135/2223] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:07.620880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.662832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:07.665227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.767869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:07.771655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122be17-6fda-4ef5-ac55-4f1e997769bd service glance] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:07.811671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1089/2224] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:07.858180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450adae-97fc-4921-98ae-e010fb8e1efe None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:07.870459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:07.872581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:07.872878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:07.875510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:07.894371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:07.894371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:07.920495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1136/2225] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:08.268990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386582d-159b-4235-b677-bd0fd83f8c31 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:08.269513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386582d-159b-4235-b677-bd0fd83f8c31 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:08.269815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386582d-159b-4235-b677-bd0fd83f8c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:08.332599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0386582d-159b-4235-b677-bd0fd83f8c31 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:08.334885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0386582d-159b-4235-b677-bd0fd83f8c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:08.545935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1090/2226] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:08.669989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:08.671547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:08.825119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188634d-b19e-46b1-bc9f-f85ffdec430a service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:08.832316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:08.832934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:08.833313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:08.833800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:08.854048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:08.865673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:08.876431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1137/2227] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:19:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:19:09.404649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:09.649949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:11.723721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:12.055107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28092a8-d585-4107-805a-55ac7366ac59 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:12.056860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28092a8-d585-4107-805a-55ac7366ac59 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:12.057489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28092a8-d585-4107-805a-55ac7366ac59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:12.101399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a28092a8-d585-4107-805a-55ac7366ac59 None None] MFA Rules not processed for user `162160bbb86e40c9bb957a213034e25f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:12.104711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a28092a8-d585-4107-805a-55ac7366ac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:12.114839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1091/2228] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:13.195676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:13.199105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:13.212580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab6e40-2546-4966-aeb5-bae6325e8de5 service swift] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:13.217904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:13.222191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:13.222191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:13.222191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:13.222191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:13.349903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:13.353820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:13.393850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1138/2229] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:14.874595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:15.090684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:19.107506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:25.562463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995156-4995-46d4-b44d-dcdd9b81afb6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:25.563247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995156-4995-46d4-b44d-dcdd9b81afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:25.563247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf995156-4995-46d4-b44d-dcdd9b81afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:25.584577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bf995156-4995-46d4-b44d-dcdd9b81afb6 None None] MFA Rules not processed for user `41bc56e0e3d0451482572959451ae043`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:25.586603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bf995156-4995-46d4-b44d-dcdd9b81afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:25.595778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1092/2230] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:25.819687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=19 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:27.139339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:27.153017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:27.154744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a493030-44e7-4d99-8fd0-3b3d01e5aa89 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:27.159604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:27.160808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:27.160929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:27.161124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] PATH_INFO: `/v3/users/19d8200cf5724bd9b647263da7a804bf` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:27.162472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d8200cf5724bd9b647263da7a804bf)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:27.165053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:27.646125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d8200cf5724bd9b647263da7a804bf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:27.653500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d8200cf5724bd9b647263da7a804bf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:27.666656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d8200cf5724bd9b647263da7a804bf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:27.667495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d8200cf5724bd9b647263da7a804bf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:27.826367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d8200cf5724bd9b647263da7a804bf'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:27.832859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1139/2231] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:27 2022] DELETE /identity/v3/users/19d8200cf5724bd9b647263da7a804bf => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:27.838284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:27.838803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:27.838976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:27.856918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:27.858149 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:27.929577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1093/2232] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:27.939692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:27.947744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:28.005147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21b72-39eb-4e96-b24e-eb19274c7536 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:28.009254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:28.009988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:28.010307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:28.010649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:28.030049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:28.031610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:28.043244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1140/2233] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:28.754925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:28.758380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:28.778244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e7748-aa42-4177-bc84-c74b635098d3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:28.785748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:28.786988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:28.786988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:28.787424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] PATH_INFO: `/v3/projects/04ddf167bad141ce9a9651bcb89d4e7f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:28.790820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ddf167bad141ce9a9651bcb89d4e7f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:28.790820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:29.027310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ddf167bad141ce9a9651bcb89d4e7f', 'request_id': 'req-a07649aa-d816-4e57-a58b-f670c98a0596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.036701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ddf167bad141ce9a9651bcb89d4e7f', 'request_id': 'req-a07649aa-d816-4e57-a58b-f670c98a0596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.045473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ddf167bad141ce9a9651bcb89d4e7f', 'request_id': 'req-a07649aa-d816-4e57-a58b-f670c98a0596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.280496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] The token cache is being invalidate because project 04ddf167bad141ce9a9651bcb89d4e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:29.280902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.281800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.284187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1094/2234] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:28 2022] DELETE /identity/v3/projects/04ddf167bad141ce9a9651bcb89d4e7f => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:29.289510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:29.301154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355c777-8531-449f-80b4-62b663e4eb03 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:29.371489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:29.372419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:29.372707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:29.372991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] PATH_INFO: `/v3/users/c39b9e278b904d3ea8188c3ae48bd7c2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:29.374387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39b9e278b904d3ea8188c3ae48bd7c2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:29.376789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:29.396485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:29.761538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39b9e278b904d3ea8188c3ae48bd7c2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.768077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39b9e278b904d3ea8188c3ae48bd7c2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.781525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39b9e278b904d3ea8188c3ae48bd7c2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:29.786354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39b9e278b904d3ea8188c3ae48bd7c2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:30.002217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0412eecc-8a3d-415b-93c5-419e6a53a4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39b9e278b904d3ea8188c3ae48bd7c2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:30.009243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1141/2235] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:29 2022] DELETE /identity/v3/users/c39b9e278b904d3ea8188c3ae48bd7c2 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:30.811527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:30.812918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:30.828674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07649aa-d816-4e57-a58b-f670c98a0596 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:30.906117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:30.907004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:30.907901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:30.908198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] PATH_INFO: `/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:30.909421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf5c1f65c2945299ad62ece75dfdd62)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:30.912846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:30.946231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:31.227034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf5c1f65c2945299ad62ece75dfdd62', 'request_id': 'req-f30353a1-a700-468e-8317-747f212e7583'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:31.234197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf5c1f65c2945299ad62ece75dfdd62', 'request_id': 'req-f30353a1-a700-468e-8317-747f212e7583'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:31.253757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf5c1f65c2945299ad62ece75dfdd62', 'request_id': 'req-f30353a1-a700-468e-8317-747f212e7583'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:31.451681 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] The token cache is being invalidate because project fbf5c1f65c2945299ad62ece75dfdd62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:31.452209 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:31.453090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f30353a1-a700-468e-8317-747f212e7583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:31.455422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1095/2236] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:30 2022] DELETE /identity/v3/projects/fbf5c1f65c2945299ad62ece75dfdd62 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:31.463968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:31.466179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:31.466179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:31.489993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:31.491421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:31.582196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1142/2237] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:31.590101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:31.590617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:31.590954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:31.625133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:31.626442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:31.640990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1096/2238] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:31.645932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:31.657005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a177474-f6ea-4f2f-af0e-1f00bf161678 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:31.661831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:31.662494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:31.672095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:31.672095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:31.672095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:19:31.672095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:31.672095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:31.681742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1143/2239] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:19:31.683815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:31.693187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c4bef-2eb3-4082-a7bb-09531d0e08ee None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:31.698115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:31.698759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:31.699051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:31.699388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:31.700100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:31.702028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:31.932565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1097/2240] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:31 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:31.939698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:31.949909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd9859-052a-4833-ba03-8c6f25c58747 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:32.018147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:32.018870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.019182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.019462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.020161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:32.022792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:32.233011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1144/2241] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:31 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:32.237250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:32.248935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590ae47-8f42-45ba-87a4-c60a564f3cd2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:32.253607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:32.254416 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.254714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.254975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.255720 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:32.258057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:32.269217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1098/2242] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:32.283432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:32.294497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540325b1-afff-46f5-9908-d7a78228a81e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:32.299596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:32.300249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.300568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.300881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.301656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:32.303914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44403b9b-958e-4979-93d7-6e79f018fdd0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:32.315769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1145/2243] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:32.321204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:32.343305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3bd1-9215-4a75-8d9c-098b03c263b9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:32.344860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:32.345879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.346269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.346647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] PATH_INFO: `/v3/projects/9cc826c751e14c91a9b001130df7ca99/users/d3aada9977dd48f188678daa4987ca47/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.364573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc826c751e14c91a9b001130df7ca99, user_id=d3aada9977dd48f188678daa4987ca47, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:32.367994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:32.498421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1099/2244] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:32 2022] PUT /identity/v3/projects/9cc826c751e14c91a9b001130df7ca99/users/d3aada9977dd48f188678daa4987ca47/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:32.513577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.514109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.514520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.545473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] MFA Rules not processed for user `d3aada9977dd48f188678daa4987ca47`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:32.546713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:32.614310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1146/2245] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:32.621192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:32.703686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05404b7-2bdc-4a38-905c-e38c37a1f264 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:32.711024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:32.713617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:32.713617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:32.713617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:32.713617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:32.716264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:32.982488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1100/2246] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:32 2022] POST /identity/v3/projects => generated 383 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:32.986560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:32.998398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d2a57-7dd5-4040-99c4-abdb4cb46901 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:33.061081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:33.061081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.061486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.061593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.062271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:33.064722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:33.226414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1147/2247] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:32 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:33.230282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:33.243501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2733f-47d6-4ad2-ba56-b6ba476fc3f0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:33.259168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:33.259939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.260285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.261388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.262147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:33.264093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:33.277782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1101/2248] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:33.282381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:33.294297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf01db-cf51-43b8-a1c2-1f73ec5de996 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:33.300526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:33.301206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.301484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.301758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.302504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:33.304662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e680833-4948-4981-9293-c7e97444f04b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:33.324251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1148/2249] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:33.326141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:33.339413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6519cc-5bc3-45f2-ac00-21ab5ccf588d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:33.344712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:33.345818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.346105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.346422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] PATH_INFO: `/v3/projects/48bbe2deec454f6084bec391e9d20004/users/c398b579a3b046698359e6836b0a90f2/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.364162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bbe2deec454f6084bec391e9d20004, user_id=c398b579a3b046698359e6836b0a90f2, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:33.376910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:33.561144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1102/2250] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:33 2022] PUT /identity/v3/projects/48bbe2deec454f6084bec391e9d20004/users/c398b579a3b046698359e6836b0a90f2/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:19:33.574335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.578893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.578893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.605727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] MFA Rules not processed for user `c398b579a3b046698359e6836b0a90f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:33.607220 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:33.695345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1149/2251] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:33.695453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:33.754429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e1db-c6b6-4b12-8087-315ac69242a0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:33.760027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:33.760700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:33.760987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:33.761266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:33.761963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:33.763964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:33.942268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1103/2252] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:33 2022] POST /identity/v3/projects => generated 383 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:33.947154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:33.958742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90189c67-a824-40ea-a790-e8eec556d0dd None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:34.017415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:34.018121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.018391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.018668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.019412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:34.021393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:34.266441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1150/2253] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:33 2022] POST /identity/v3/users => generated 298 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:34.276042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:34.288023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85874ba3-5bfe-4201-b7c5-20cb002a786d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:34.293412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:34.294197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.294604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.294991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.295831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:34.297638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:34.311045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1104/2254] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:34.315532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:34.326538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f81260-37b5-49c7-a73f-d8c932f2ebcb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:34.334014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:34.340710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.340710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.340710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] PATH_INFO: `/v3/projects/16920a1eb8964410b2de660f759de6e9/users/6c2d7ec0cbbc487bb5dee356b0f116b7/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.355624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16920a1eb8964410b2de660f759de6e9, user_id=6c2d7ec0cbbc487bb5dee356b0f116b7, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:34.358695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:34.517265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1151/2255] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:34 2022] PUT /identity/v3/projects/16920a1eb8964410b2de660f759de6e9/users/6c2d7ec0cbbc487bb5dee356b0f116b7/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:34.521553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:34.533179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c7d9-fbcb-4628-8ee1-a2390a967bf2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:34.607261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e36e7-329a-4556-9c5a-dcf28e4b2380 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:34.625624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1105/2256] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:34.632168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:34.642940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831963c4-26bf-4a1a-9c57-79593a785b86 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:34.649530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:34.650426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.650710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.650985 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] PATH_INFO: `/v3/projects/16920a1eb8964410b2de660f759de6e9/users/6c2d7ec0cbbc487bb5dee356b0f116b7/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.653525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16920a1eb8964410b2de660f759de6e9, user_id=6c2d7ec0cbbc487bb5dee356b0f116b7, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:34.656368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:34.669679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1152/2257] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:34 2022] PUT /identity/v3/projects/16920a1eb8964410b2de660f759de6e9/users/6c2d7ec0cbbc487bb5dee356b0f116b7/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:34.686730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3958e2-0664-45f2-ab4a-9d85c82a585a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:34.687046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3958e2-0664-45f2-ab4a-9d85c82a585a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:34.687369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3958e2-0664-45f2-ab4a-9d85c82a585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:34.717313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9f3958e2-0664-45f2-ab4a-9d85c82a585a None None] MFA Rules not processed for user `6c2d7ec0cbbc487bb5dee356b0f116b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:34.718777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9f3958e2-0664-45f2-ab4a-9d85c82a585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:34.813910 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:34.821422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1106/2258] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:34.839187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:34.841959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:35.034192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae718c8-e55f-405d-86b1-c7588843af5a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:35.038496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:35.039175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:35.039461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:35.039736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:35.096512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:35.097949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:35.107157 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1153/2259] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:36.376289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:38.833022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:38.833641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:38.833641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:38.860866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:38.863252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:39.014846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:39.015430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1107/2260] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:19:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:39.026326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301a6ac-e167-4acd-9f93-4b040be94451 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:39.033257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:39.034209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:39.034595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:39.034977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:39.035172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:39.039343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:39.151467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:39.166813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:39.169746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:39.188567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1154/2261] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:39.212181 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ba849-cfb1-4135-a43d-340f2cabed90 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:39.222288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:39.223398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:39.224192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:39.225243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:39.237120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:39.239007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe512e-993a-4c33-b0dc-d9cb27e8cb90 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:39.251743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1108/2262] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:19:41.262026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:42.017636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:42.031689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b7686-e217-44ec-aad1-e9bbf70e5501 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:42.039626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:42.042383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:42.043380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:42.044071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:42.057817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:42.060455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:42.074498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1155/2263] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:42.669544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:46.119432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:46.119432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:46.119432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:46.144027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] MFA Rules not processed for user `e3e8aa5fc932453e95080fa836dfac0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:46.144027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:46.226500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1109/2264] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:19:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:55.436962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=32 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:56.216109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=87 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:04.135002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:04.136819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:04.253649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcda872-8d5f-4c6c-92b9-54486153b196 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:04.260317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:04.261575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:04.261876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:04.262158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:04.264826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:04.280523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:04.282441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc87ebb-86f0-4798-92ce-8e6b2cae6509 service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:04.303832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1156/2265] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:08.238130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:08.308562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb52009-e35f-4cb8-b95c-141fa2548163 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:08.313820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:08.314700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:08.318990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:08.318990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] PATH_INFO: `/v3/users/d3aada9977dd48f188678daa4987ca47` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:08.318990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3aada9977dd48f188678daa4987ca47)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:08.319428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:08.594203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3aada9977dd48f188678daa4987ca47'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:08.595080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3aada9977dd48f188678daa4987ca47'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:08.614941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3aada9977dd48f188678daa4987ca47'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:08.625415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3aada9977dd48f188678daa4987ca47'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:08.763669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3aada9977dd48f188678daa4987ca47'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:08.780313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1110/2266] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:08 2022] DELETE /identity/v3/users/d3aada9977dd48f188678daa4987ca47 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:08.799368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:08.800224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:08.800369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:08.828595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:08.829770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:08.914155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1157/2267] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:08.924059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:08.925494 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:08.999266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f4d8b-20a5-4772-a063-f32d2bf15ee7 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:09.006747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:09.007983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:09.007983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:09.007983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:09.023888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:09.026682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:09.043811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1111/2268] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:09.800417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:09.806451 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:09.814744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5827a5-f6f2-4baf-ae4a-6176e08dcba1 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:09.820626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:09.821489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:09.821805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:09.822101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] PATH_INFO: `/v3/projects/9cc826c751e14c91a9b001130df7ca99` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:09.823864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc826c751e14c91a9b001130df7ca99)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:09.825774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:09.970863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc826c751e14c91a9b001130df7ca99', 'request_id': 'req-7c351f4c-7731-455a-87fd-375ee901161e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:09.976317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc826c751e14c91a9b001130df7ca99', 'request_id': 'req-7c351f4c-7731-455a-87fd-375ee901161e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.067215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc826c751e14c91a9b001130df7ca99', 'request_id': 'req-7c351f4c-7731-455a-87fd-375ee901161e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.084540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] The token cache is being invalidate because project 9cc826c751e14c91a9b001130df7ca99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:10.085019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.085880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.087862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1158/2269] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:09 2022] DELETE /identity/v3/projects/9cc826c751e14c91a9b001130df7ca99 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:10.091952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:10.103432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d314720-c026-4986-8320-0c5ad13e87aa service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:10.125655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:10.171271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:10.172581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:10.173071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:10.174655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] PATH_INFO: `/v3/users/c398b579a3b046698359e6836b0a90f2` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:10.179752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c398b579a3b046698359e6836b0a90f2)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:10.184785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:10.390210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c398b579a3b046698359e6836b0a90f2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.390939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c398b579a3b046698359e6836b0a90f2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.405308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c398b579a3b046698359e6836b0a90f2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.412552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c398b579a3b046698359e6836b0a90f2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.483787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c398b579a3b046698359e6836b0a90f2'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:10.490331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1112/2270] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:10 2022] DELETE /identity/v3/users/c398b579a3b046698359e6836b0a90f2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:11.234211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:11.235794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:11.250101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c351f4c-7731-455a-87fd-375ee901161e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:11.271098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:11.333829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:11.333829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:11.333829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:11.333829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] PATH_INFO: `/v3/projects/48bbe2deec454f6084bec391e9d20004` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:11.334205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bbe2deec454f6084bec391e9d20004)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:11.334205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:11.451518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bbe2deec454f6084bec391e9d20004', 'request_id': 'req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.457358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bbe2deec454f6084bec391e9d20004', 'request_id': 'req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.535675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bbe2deec454f6084bec391e9d20004', 'request_id': 'req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.547347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] The token cache is being invalidate because project 48bbe2deec454f6084bec391e9d20004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:11.547633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.548254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.549778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1159/2271] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:11 2022] DELETE /identity/v3/projects/48bbe2deec454f6084bec391e9d20004 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:11.554993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:11.556591 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:11.570208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad46ce0-f9c5-4345-8752-d191b71afa8e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:11.629347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:11.630239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:11.630525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:11.630795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] PATH_INFO: `/v3/users/6c2d7ec0cbbc487bb5dee356b0f116b7` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:11.632696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2d7ec0cbbc487bb5dee356b0f116b7)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:11.634848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:11.798848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:11.801426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2d7ec0cbbc487bb5dee356b0f116b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.802076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2d7ec0cbbc487bb5dee356b0f116b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.815396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2d7ec0cbbc487bb5dee356b0f116b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.823997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2d7ec0cbbc487bb5dee356b0f116b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.933923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2d7ec0cbbc487bb5dee356b0f116b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:11.940664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1113/2272] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:11 2022] DELETE /identity/v3/users/6c2d7ec0cbbc487bb5dee356b0f116b7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:20:12.446224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:12.448435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:12.464104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268f9b9-27e4-4ad7-b482-9ea6f57897a1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:12.528523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:12.529433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:12.529715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:12.529993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] PATH_INFO: `/v3/projects/16920a1eb8964410b2de660f759de6e9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:12.531318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=16920a1eb8964410b2de660f759de6e9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:12.533237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:12.642787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:12.645720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16920a1eb8964410b2de660f759de6e9', 'request_id': 'req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:12.651808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16920a1eb8964410b2de660f759de6e9', 'request_id': 'req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:12.770512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16920a1eb8964410b2de660f759de6e9', 'request_id': 'req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:12.785188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] The token cache is being invalidate because project 16920a1eb8964410b2de660f759de6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:12.785750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:12.787078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dfddb76a-6f51-4ffa-a06d-b9f2ebaeff8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:12.788968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1160/2273] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:12 2022] DELETE /identity/v3/projects/16920a1eb8964410b2de660f759de6e9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:16.466169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:16.466169 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:16.480635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0565-6d7a-46a8-9247-6ecb7b7e259a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:16.550640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:16.551555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:16.551957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:16.552096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] PATH_INFO: `/v3/users/47a4727579ef4c439e6b0d5c56ebef01` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:16.553370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a4727579ef4c439e6b0d5c56ebef01)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:16.557050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:16.850129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a4727579ef4c439e6b0d5c56ebef01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:16.850885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a4727579ef4c439e6b0d5c56ebef01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:16.874158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a4727579ef4c439e6b0d5c56ebef01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:16.881670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a4727579ef4c439e6b0d5c56ebef01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:16.994736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a4727579ef4c439e6b0d5c56ebef01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:17.002566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1114/2274] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:16 2022] DELETE /identity/v3/users/47a4727579ef4c439e6b0d5c56ebef01 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:17.007394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:17.007955 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:17.008484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:17.026952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:17.028166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:17.105912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1161/2275] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:17.115172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:17.116705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:17.162479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:17.188935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c33232-f0a3-4801-9409-65c56e350661 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:17.192918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:17.200536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:17.200536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:17.200536 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:17.205471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:17.206943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54402965-55a7-4e7a-bd0e-8e46998e6785 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:17.220899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1115/2276] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:17.940163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:17.941975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:17.955058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beeafb-5604-42c7-b9b8-1aa2cd4aeecf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:17.960305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:17.962179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:17.962752 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:17.963105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] PATH_INFO: `/v3/projects/2eb05bae9b0b403c8834967ed1d689ab` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:17.965197 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb05bae9b0b403c8834967ed1d689ab)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:17.967259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:18.174143 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb05bae9b0b403c8834967ed1d689ab', 'request_id': 'req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:18.197708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb05bae9b0b403c8834967ed1d689ab', 'request_id': 'req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:18.280841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb05bae9b0b403c8834967ed1d689ab', 'request_id': 'req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:18.299743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] The token cache is being invalidate because project 2eb05bae9b0b403c8834967ed1d689ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:18.300054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:18.300724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5df06942-8ec1-4ebc-9a22-22a6f6c5c0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:18.302298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1162/2277] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:17 2022] DELETE /identity/v3/projects/2eb05bae9b0b403c8834967ed1d689ab => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:20:18.312146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:18.333917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:18.333917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:18.335575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:18.336965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:18.449814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1116/2278] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:18.451241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:18.451535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:18.451822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:18.488542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:18.489871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:18.519802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1163/2279] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:18.521554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:18.533214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee85fd-7973-4f77-9863-69930b7e34fd None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:18.540301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:18.541635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:18.542027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:18.542403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:18.543310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:20:18.543730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:18.545691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:18.564390 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1117/2280] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:20:18.593225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:18.604152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef86a-041f-4940-b0a2-6979c050aa00 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:18.612056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:18.612164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:18.612709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:18.612820 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:18.614117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:18.616890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:18.743674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1164/2281] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:18 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:18.747932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:18.761873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05321c90-b6dc-4cb0-95a0-c260465252bc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:18.865196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:18.865817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:18.866088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:18.866367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:18.868116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:18.870176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:19.003144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1118/2282] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:20:18 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:19.007683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:19.026042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bbe99-a426-4cd3-8211-a9560f2a1648 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:19.041696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:19.044241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:19.044813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:19.045108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:19.045908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:19.048379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:19.061997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1165/2283] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:19.072405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:19.097942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3675e1-259c-40f1-85ac-240e2dea3fc0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:19.108180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:19.108968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:19.113489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:19.113489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:19.113489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:19.114412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d4162-939d-4c66-a6d7-cd2215f91203 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:19.136485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1119/2284] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:19.143050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:19.171085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a63f57-5c35-43b3-9b8f-4bb001d87a12 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:19.187618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:19.193004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:19.193771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:19.194373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] PATH_INFO: `/v3/projects/11215c9c5f59498c8110024722b1529c/users/ee2e478cdbb04617b48f388a4b38bd35/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:19.218828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11215c9c5f59498c8110024722b1529c, user_id=ee2e478cdbb04617b48f388a4b38bd35, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:19.222746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:19.225831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:19.368333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1166/2285] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:20:19 2022] PUT /identity/v3/projects/11215c9c5f59498c8110024722b1529c/users/ee2e478cdbb04617b48f388a4b38bd35/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:19.369955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:19.370521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:19.370521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:19.396091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:19.397423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:19.456526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:19.478184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1120/2286] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:19.491333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:19.495067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:19.570104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312758c-f9b4-4326-92b7-fa8b9f66f983 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:19.578005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:19.579793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:19.579908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:19.580395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:19.596690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:19.599321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1b76c-bffb-4ed5-8509-d2bf9b40f241 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:19.620031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1167/2287] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:21.841869 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:21.843701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:21.857406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185b6e3-bce0-44f0-ae43-fbc30ad51695 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:21.862379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:21.863361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:21.863652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:21.863934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] PATH_INFO: `/v3/users/48d795546ed24bf784051480712ef2d3` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:21.865214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d795546ed24bf784051480712ef2d3)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:21.867229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:22.157151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d795546ed24bf784051480712ef2d3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:22.157574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d795546ed24bf784051480712ef2d3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:22.180096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d795546ed24bf784051480712ef2d3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:22.190748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d795546ed24bf784051480712ef2d3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:22.260850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e208-2df8-4c6a-97e1-9182ed8d7924 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.261575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e208-2df8-4c6a-97e1-9182ed8d7924 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.261575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e208-2df8-4c6a-97e1-9182ed8d7924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.312693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-09c1e208-2df8-4c6a-97e1-9182ed8d7924 None None] MFA Rules not processed for user `ee2e478cdbb04617b48f388a4b38bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:22.313967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-09c1e208-2df8-4c6a-97e1-9182ed8d7924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.413753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d795546ed24bf784051480712ef2d3'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:22.420964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1168/2288] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:22.423682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1121/2289] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:21 2022] DELETE /identity/v3/users/48d795546ed24bf784051480712ef2d3 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:22.435813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.436069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.436915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.458113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:22.459417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.459615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:22.461618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.534392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1169/2290] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:22.550841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:22.552638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.556381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f151d5e-c07b-4a26-b73a-c0e30586a564 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:22.560328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:22.560985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.561271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.561547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.587367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:22.616117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8573ff9-d493-4257-9367-7b57d06d76cc None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:22.621118 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:22.621809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.622150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.622425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.625713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:22.627312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:22.634679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:22.637152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716c99-9e17-4da0-8eaf-4debf99911b6 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:22.640219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1122/2291] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:20:22.653319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1170/2292] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:23.188121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:23.189598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:23.213300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdb1e7-393d-4e45-9c81-6095974c663c service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:23.213300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:23.213912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:23.214196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:23.214481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] PATH_INFO: `/v3/projects/b579dcf22a78460eba58bc42e539f8c6` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:23.216114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b579dcf22a78460eba58bc42e539f8c6)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:23.218309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:23.234247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:23.387144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b579dcf22a78460eba58bc42e539f8c6', 'request_id': 'req-6b842782-af6b-4932-9042-738fc7c825ce'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:23.392743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b579dcf22a78460eba58bc42e539f8c6', 'request_id': 'req-6b842782-af6b-4932-9042-738fc7c825ce'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:23.399071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b579dcf22a78460eba58bc42e539f8c6', 'request_id': 'req-6b842782-af6b-4932-9042-738fc7c825ce'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:23.544071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] The token cache is being invalidate because project b579dcf22a78460eba58bc42e539f8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:23.544654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:23.546018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6b842782-af6b-4932-9042-738fc7c825ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:23.548862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1123/2293] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:23 2022] DELETE /identity/v3/projects/b579dcf22a78460eba58bc42e539f8c6 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:23.561698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:23.562599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:23.563318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:23.592207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:23.593759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:23.705668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1171/2294] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:23.719459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:23.719919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:23.720415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:23.756643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:23.757926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:23.770964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1124/2295] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:23.773029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:23.784886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574f9ea-25d6-48ea-b395-3bc78bf14586 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:23.790921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:23.792044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:23.792402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:23.792778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:23.796051 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:20:23.796378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:23.798247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:23.820790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1172/2296] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:20:23.824158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:23.835842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ffb-bf40-4acc-a605-84a3eedf860e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:23.841092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:23.841789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:23.842076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:23.842353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:23.843094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:23.845109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:23.986231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1125/2297] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:23 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:23.992263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:24.003766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9c80-a485-44d8-a100-5428f67f9fbd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:24.096095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:24.096774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.097083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.097356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.098042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:24.100042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:24.279944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1173/2298] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:20:23 2022] POST /identity/v3/users => generated 288 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:24.286016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:24.300678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7645a-0253-460b-9c8a-e5a876ac7c57 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:24.306866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:24.307700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.308064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.308357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.309249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:24.311507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:24.327781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1126/2299] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:24.331813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:24.344724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9744f08-58fa-45c3-ac2f-2588a1d08d66 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:24.349989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:24.350934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.351327 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.351800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.354431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:24.357958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367b1a8-158d-422d-987c-a921a4cc64b2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:24.372270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1174/2300] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:20:24.377855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:24.393696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982badd-2c96-485f-90ac-5b966a7ee068 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:24.402245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:24.403857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.404447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.404880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] PATH_INFO: `/v3/projects/fff6b9f656b24b4a8b5724f9a0078b2f/users/6a5a5aa9ccde4aa0ac0362b11010f5c0/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.436859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff6b9f656b24b4a8b5724f9a0078b2f, user_id=6a5a5aa9ccde4aa0ac0362b11010f5c0, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:24.441834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:24.576500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1127/2301] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:20:24 2022] PUT /identity/v3/projects/fff6b9f656b24b4a8b5724f9a0078b2f/users/6a5a5aa9ccde4aa0ac0362b11010f5c0/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:24.589554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9823b-af0e-4c6b-b66c-cee2270eb164 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.590439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9823b-af0e-4c6b-b66c-cee2270eb164 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.590439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9823b-af0e-4c6b-b66c-cee2270eb164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.627894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-eec9823b-af0e-4c6b-b66c-cee2270eb164 None None] MFA Rules not processed for user `6a5a5aa9ccde4aa0ac0362b11010f5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:24.628428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-eec9823b-af0e-4c6b-b66c-cee2270eb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:24.653348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:24.721361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1175/2302] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:24.731294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:24.732236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:24.821518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888e835-57a8-419b-93ff-c86706a4f72f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:24.826409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:24.827108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:24.827655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:24.828159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:24.843097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:24.845065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb37b-1704-4de2-acf0-14d76b6a16fd service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:24.856328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1128/2303] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:26.008826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596512-bb99-4a95-99b1-c8068d4fcec2 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:26.009336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596512-bb99-4a95-99b1-c8068d4fcec2 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:26.009959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596512-bb99-4a95-99b1-c8068d4fcec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:26.031638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bc596512-bb99-4a95-99b1-c8068d4fcec2 None None] MFA Rules not processed for user `ee2e478cdbb04617b48f388a4b38bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:26.032846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc596512-bb99-4a95-99b1-c8068d4fcec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:26.098087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1176/2304] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:28.192738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:28.195741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:28.195741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:28.214864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] MFA Rules not processed for user `6a5a5aa9ccde4aa0ac0362b11010f5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:28.216400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:28.226002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1129/2305] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:34.305938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eb5d1-f41b-4b41-91c0-f0d0e73f658e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:34.306567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eb5d1-f41b-4b41-91c0-f0d0e73f658e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:34.306567 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3eb5d1-f41b-4b41-91c0-f0d0e73f658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:34.327999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3e3eb5d1-f41b-4b41-91c0-f0d0e73f658e None None] MFA Rules not processed for user `162160bbb86e40c9bb957a213034e25f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:34.329858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3e3eb5d1-f41b-4b41-91c0-f0d0e73f658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:34.412662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1177/2306] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:20:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:37.692933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:37.694372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:37.744366 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:37.787818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb4fd-942f-47e8-97d2-ebd0676acd13 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:37.798168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:37.800011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:37.800796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:37.801596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:37.805808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:37.931319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:37.934747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:37.953071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1130/2307] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:44.404164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=66 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:48.408480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=36 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:05.663014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191504-2dcd-4c04-90f1-9ee9d20bef7d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:05.663967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191504-2dcd-4c04-90f1-9ee9d20bef7d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:05.663967 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55191504-2dcd-4c04-90f1-9ee9d20bef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:05.686569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-55191504-2dcd-4c04-90f1-9ee9d20bef7d None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:05.687864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-55191504-2dcd-4c04-90f1-9ee9d20bef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:05.756228 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1178/2308] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:05.805868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:05.807598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:05.934941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac4d8-726e-4bcf-89a0-8c82b6d6e9af service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:05.951463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:05.951463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:05.951463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:05.951463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:05.963066 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:05.964709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:05.978516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1131/2309] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:21:07.330504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97bb8-f749-42a0-98e9-ea55a06b4ffc None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:07.331314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97bb8-f749-42a0-98e9-ea55a06b4ffc None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:07.331926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef97bb8-f749-42a0-98e9-ea55a06b4ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:07.363774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-3ef97bb8-f749-42a0-98e9-ea55a06b4ffc None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:07.365009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3ef97bb8-f749-42a0-98e9-ea55a06b4ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:07.377120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1179/2310] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:21:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:07.393902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:07.405470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09e87-c141-4dc8-9427-52bfec3af969 service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:07.412945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:07.414045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:07.414497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:07.414954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:07.428682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:07.430621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:07.444028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1132/2311] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:21:08.397299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6765e3-9206-4002-b2ab-131ff088ffba None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:08.398178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6765e3-9206-4002-b2ab-131ff088ffba None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:08.398178 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6765e3-9206-4002-b2ab-131ff088ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:08.430884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bc6765e3-9206-4002-b2ab-131ff088ffba None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:08.433377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bc6765e3-9206-4002-b2ab-131ff088ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:08.444603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1180/2312] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:21:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:21:08.462874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:08.482548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529b702-7375-4c96-a2b3-fb21a18b32ea service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:08.489217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:08.489902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:08.490192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:08.490482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:08.500493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:08.503063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:08.513035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1133/2313] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:21:09.317590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:09.318153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:09.318875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:09.347086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:09.349895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:09.363356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1181/2314] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:21:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:09.384136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:09.403976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3125a8-e6be-493e-bf25-72ae5fb3b7ee service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:09.413490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:09.414758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:09.415266 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:09.415672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:09.433064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:09.436221 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205147f-7a12-4b49-86c6-a5ed04f72182 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:09.450601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1134/2315] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:21:13.065714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:13.185780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a54cc8-acc3-445d-aa69-5ff1c17cb095 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:13.193628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:13.194677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:13.194840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:13.195190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] PATH_INFO: `/v3/users/6a5a5aa9ccde4aa0ac0362b11010f5c0` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:13.196626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5a5aa9ccde4aa0ac0362b11010f5c0)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:13.215573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:13.478119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5a5aa9ccde4aa0ac0362b11010f5c0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:13.490369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5a5aa9ccde4aa0ac0362b11010f5c0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:13.506056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5a5aa9ccde4aa0ac0362b11010f5c0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:13.506828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5a5aa9ccde4aa0ac0362b11010f5c0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:13.656530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5a5aa9ccde4aa0ac0362b11010f5c0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:13.668233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1182/2316] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:21:13 2022] DELETE /identity/v3/users/6a5a5aa9ccde4aa0ac0362b11010f5c0 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:13.676138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:13.677850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:13.678374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:13.724571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:13.726126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:13.850043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1135/2317] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:13.862563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:13.864117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:13.872448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:13.991815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1c51c-082a-407f-89be-0c35983f565d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:13.995705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:13.996369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:13.996655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:13.996930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:14.008325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:14.009944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0eea2-4f4f-4797-aa0e-538549a29c34 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:14.028198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1183/2318] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:21:14.446133 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:14.450046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:14.471706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd7ff-4f92-4b2a-85bc-dbabfd4b59a8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:14.480480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:14.496789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:14.497259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:14.497593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] PATH_INFO: `/v3/projects/fff6b9f656b24b4a8b5724f9a0078b2f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:14.520290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff6b9f656b24b4a8b5724f9a0078b2f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:14.523055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:14.716839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:14.718519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff6b9f656b24b4a8b5724f9a0078b2f', 'request_id': 'req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:14.727737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff6b9f656b24b4a8b5724f9a0078b2f', 'request_id': 'req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:14.735724 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff6b9f656b24b4a8b5724f9a0078b2f', 'request_id': 'req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:14.986699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] The token cache is being invalidate because project fff6b9f656b24b4a8b5724f9a0078b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:21:14.987419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:14.988854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-ef5b14a0-efcc-427d-9d6c-bf75c4443b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:21:14.995429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1136/2319] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:21:14 2022] DELETE /identity/v3/projects/fff6b9f656b24b4a8b5724f9a0078b2f => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:15.000174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.000648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.000648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.025254 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:15.026463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:15.117383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1184/2320] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:15.136178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.136178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.136178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.158997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:15.160693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:15.173876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1137/2321] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:15.178277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.192365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d33ec4-94b4-4550-a04c-73ed622ed27c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.199522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.200546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.201016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.201532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.203059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:21:15.203628 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.207518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.230257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1185/2322] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:21:15.235463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.250995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f05a34-3dca-43d5-84df-f8f38d7c82e7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.260337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.260809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.261259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.261670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.262735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.266217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.392437 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1138/2323] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:21:15 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:15.398623 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.416347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2fdc-dffe-4354-8650-fd0ae0a550fe admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.488868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.489966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.490347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.490630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.491410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.494231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.588019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1186/2324] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:21:15 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:15.594605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.612912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec9ead-2078-4afe-a0f4-7cd8cc8c2902 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.621529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.622961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.623686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.624197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.625656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.630805 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.646970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1139/2325] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:21:15.651846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.664667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d883d50-33f7-474f-a584-b73831256248 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.673137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.675105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.675773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.676377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.677706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.679810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b08536-c21c-4245-a5bc-2397a5564957 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.694459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1187/2326] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:21:15.700272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:15.717243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83edc6e-8d15-47e4-8b64-a775fbbd6709 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:15.725047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:15.726443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.726886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.727407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] PATH_INFO: `/v3/projects/81b8c45981fe4d48bf95dfbdd0804205/users/6b84187a669e4849b3b9d02d8b337824/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.749028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b8c45981fe4d48bf95dfbdd0804205, user_id=6b84187a669e4849b3b9d02d8b337824, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:15.751985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:15.868041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1140/2327] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:21:15 2022] PUT /identity/v3/projects/81b8c45981fe4d48bf95dfbdd0804205/users/6b84187a669e4849b3b9d02d8b337824/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:15.874344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20fd0e-2bd8-4039-9fa2-13d1db298b2a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:15.874835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20fd0e-2bd8-4039-9fa2-13d1db298b2a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:15.875360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20fd0e-2bd8-4039-9fa2-13d1db298b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:15.904713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4e20fd0e-2bd8-4039-9fa2-13d1db298b2a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:15.906675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e20fd0e-2bd8-4039-9fa2-13d1db298b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:15.921997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:15.993789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1188/2328] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:16.007731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:16.010587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:16.086736 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4cca-0cd6-4949-b18e-f2367bd26dcf admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:16.092116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:16.092757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:16.093055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:16.093344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:16.106161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:16.107767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:16.119529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1141/2329] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:21:22.717946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:22.718457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:22.718457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:22.746835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] MFA Rules not processed for user `6b84187a669e4849b3b9d02d8b337824`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:22.748045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:22.817239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1189/2330] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:22.821560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:22.833959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c67e-55f5-469e-b64c-9dd26f3a8264 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:22.839041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:22.839755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:22.840042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:22.840319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:22.841012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:22.842939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:22.981252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1142/2331] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:21:22 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:22.985290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:22.996139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9076cf1-dcf7-4f6f-89d2-ad860cd80710 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:23.065248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:23.065960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:23.066241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:23.066516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:23.067229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:23.069159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:23.186507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1190/2332] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:21:22 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:23.192436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:23.207823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf00e2d-bddb-4978-9bec-27de4aa73179 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:23.214753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:23.215488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:23.215776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:23.216057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:23.216806 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:23.218432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:23.233654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1143/2333] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:21:23.238250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:23.249336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdaf49-0491-48c7-84e4-70beed5e193a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:23.254347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:23.255342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:23.255625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:23.255898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] PATH_INFO: `/v3/projects/b23c451f459944fdb64f2a8edd443d07/users/0d9974b0e7984c118c57bee5104f756b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:23.272123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23c451f459944fdb64f2a8edd443d07, user_id=0d9974b0e7984c118c57bee5104f756b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:23.275701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:23.360151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1191/2334] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:21:23 2022] PUT /identity/v3/projects/b23c451f459944fdb64f2a8edd443d07/users/0d9974b0e7984c118c57bee5104f756b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:23.364412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:23.375781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d8d7f-3ea8-4ee5-93e6-671d3522a91a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:23.418475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:23.439941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:23.441217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:23.441497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:23.441787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:23.442622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:23.444281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ab9da-f07a-46e0-b9b2-1de65d41fc30 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:23.455335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1144/2335] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:21:23.460540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:23.471738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588645ed-3b40-4720-a535-33b87728fa1f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:23.479489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:23.480398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:23.480684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:23.480965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] PATH_INFO: `/v3/projects/b23c451f459944fdb64f2a8edd443d07/users/0d9974b0e7984c118c57bee5104f756b/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:23.482961 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23c451f459944fdb64f2a8edd443d07, user_id=0d9974b0e7984c118c57bee5104f756b, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:23.485694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:23.499367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:23.508285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1192/2336] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:21:23 2022] PUT /identity/v3/projects/b23c451f459944fdb64f2a8edd443d07/users/0d9974b0e7984c118c57bee5104f756b/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:28.232991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b3d2-e477-4da0-b6fe-aa7feb109315 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:28.233545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b3d2-e477-4da0-b6fe-aa7feb109315 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:28.233545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894b3d2-e477-4da0-b6fe-aa7feb109315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:28.264770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a894b3d2-e477-4da0-b6fe-aa7feb109315 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:28.266427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a894b3d2-e477-4da0-b6fe-aa7feb109315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:28.352160 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1145/2337] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:21:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:28.376101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:28.380467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:28.502323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3da1b-8393-4b7b-9f4a-86d7fd5d21b6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:28.510548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:28.511385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:28.511599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:28.512021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:28.524568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:28.526506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:28.547871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1193/2338] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:21:31.368168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbdf64-f7d1-4435-9a36-dd96ec5c112c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:31.368168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbdf64-f7d1-4435-9a36-dd96ec5c112c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:31.369029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbdf64-f7d1-4435-9a36-dd96ec5c112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:31.418495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d9bbdf64-f7d1-4435-9a36-dd96ec5c112c None None] MFA Rules not processed for user `0d9974b0e7984c118c57bee5104f756b`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:31.419909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d9bbdf64-f7d1-4435-9a36-dd96ec5c112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:31.531831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1146/2339] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:31.546556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:31.548075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:31.677606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263c4dc-0013-498f-b107-34d297d75c2c service swift] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:31.689321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:31.690367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:31.690868 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:31.691443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:31.834687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:31.838713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6ae84-9368-4259-b75e-a3a7ede45a44 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:31.871865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1194/2340] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:21:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:21:38.073664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:38.073664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:38.073664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:38.109873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] MFA Rules not processed for user `6b84187a669e4849b3b9d02d8b337824`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:38.112898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:38.124041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1147/2341] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:21:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:41.861512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:48.094710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=73 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:56.344543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27764f-9f83-40e1-bd01-97048e400a94 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:56.345148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27764f-9f83-40e1-bd01-97048e400a94 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:56.345148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27764f-9f83-40e1-bd01-97048e400a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:56.367509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0f27764f-9f83-40e1-bd01-97048e400a94 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:56.368783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0f27764f-9f83-40e1-bd01-97048e400a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:56.376957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1195/2342] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:56.392110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:56.408875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16b19-c087-4311-b276-550c80e7ae95 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:56.415215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:56.416010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:56.416300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:56.416574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:56.439276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:56.442499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:56.460797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1148/2343] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:21:58.733997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:58.734982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:58.735345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:58.755987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:58.758213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:58.767624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1196/2344] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:21:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:58.780516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:58.792787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550a0ad-1b84-4b3d-a098-1d05cab467b3 service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:58.798172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:58.811178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:58.811178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:58.811178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:58.811178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:58.811178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:58.822485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1149/2345] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:22:00.751044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:00.859374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7ab3f-b853-408a-a1a0-ecc2e7f170fe None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:00.864038 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:00.864895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:00.865183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:00.865459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:00.869457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:00.887270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:00.889607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:00.907683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1197/2346] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:02.151350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:02.152696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:02.167131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4597f-927a-4080-8162-f15b2c2e6861 service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:02.171424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:02.172074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:02.172360 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:02.172642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:02.174452 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:02.191382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:02.192841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:02.206469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1150/2347] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:09.418301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:09.419860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:09.547077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ceda-acc0-465e-a4c4-dea9b2bd27d3 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:09.554711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:09.557082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:09.557605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:09.558113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] PATH_INFO: `/v3/users/162160bbb86e40c9bb957a213034e25f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:09.560532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162160bbb86e40c9bb957a213034e25f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:09.562751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:09.922731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162160bbb86e40c9bb957a213034e25f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:09.939630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162160bbb86e40c9bb957a213034e25f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:09.981217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162160bbb86e40c9bb957a213034e25f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:09.983044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162160bbb86e40c9bb957a213034e25f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.133743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-66044ee4-4a2e-4fc1-96fe-104188716851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162160bbb86e40c9bb957a213034e25f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.142358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1198/2348] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:22:09 2022] DELETE /identity/v3/users/162160bbb86e40c9bb957a213034e25f => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:10.498361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:10.525853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d414c0-e287-4b1e-a098-9b452e6dde0b service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:10.632732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:10.634552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:10.634933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:10.635666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] PATH_INFO: `/v3/projects/76d33745da9045b989026dad3b5b7abb` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:10.637986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d33745da9045b989026dad3b5b7abb)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:10.641657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:10.793842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d33745da9045b989026dad3b5b7abb', 'request_id': 'req-08593bf1-204c-400a-af6d-de575b27fd16'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.804694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d33745da9045b989026dad3b5b7abb', 'request_id': 'req-08593bf1-204c-400a-af6d-de575b27fd16'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.815289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d33745da9045b989026dad3b5b7abb', 'request_id': 'req-08593bf1-204c-400a-af6d-de575b27fd16'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.900323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] The token cache is being invalidate because project 76d33745da9045b989026dad3b5b7abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:22:10.900663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.901738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-08593bf1-204c-400a-af6d-de575b27fd16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:10.905030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1151/2349] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:22:10 2022] DELETE /identity/v3/projects/76d33745da9045b989026dad3b5b7abb => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:10.913446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:10.914360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:10.914360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:10.964943 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:10.966780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:10.972689 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:11.104056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1199/2350] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:22:11.113785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.114171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.114578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.144407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:11.146045 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:11.168684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1152/2351] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:22:11.172924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.185877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e555b-6821-42ab-8554-c7f065ec869d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.196164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.197172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.197463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.197746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.198531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:22:11.198846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.200758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:11.217080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1200/2352] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:22:11.221888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.242110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3370b1-400f-455e-8d8b-ee1de4bcf3f6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.247566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.248797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.249163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.249449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.250154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.252127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:11.594838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1153/2353] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:22:11 2022] POST /identity/v3/projects => generated 385 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:22:11.601192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.611552 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d237f1-7d08-49c4-bb49-59b2436a0e08 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.668398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.669068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.669350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.669629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.670326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.672265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:11.820253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1201/2354] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:22:11 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:22:11.825346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.837061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c56f70-9b0f-40ed-85b1-304290e7d64a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.842804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.843503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.843789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.844067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.844890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.846615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:11.858733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1154/2355] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:22:11.866303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.878472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c056c2-efcb-4628-b674-de62962610e3 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.885892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.888500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.888792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.889074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.889812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.891471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b95731-965a-4758-82a4-76abceb0ec1d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:11.903304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1202/2356] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:22:11.907689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:11.917845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa9d7b-09da-4b2a-9e32-53bcf798d4bb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:11.923946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:11.924877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:11.925174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:11.925466 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] PATH_INFO: `/v3/projects/62facb800416476a9e42bd6e2bd03a18/users/5246e2b75484450881ec8c9056134536/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:11.936240 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:11.943453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62facb800416476a9e42bd6e2bd03a18, user_id=5246e2b75484450881ec8c9056134536, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:11.946723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:12.111443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1155/2357] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:22:11 2022] PUT /identity/v3/projects/62facb800416476a9e42bd6e2bd03a18/users/5246e2b75484450881ec8c9056134536/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:12.122864 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:12.123417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:12.123540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:12.152360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] MFA Rules not processed for user `5246e2b75484450881ec8c9056134536`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:12.153615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:12.280874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1203/2358] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:22:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:22:12.294361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:12.296831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:12.423738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0bcb5-e031-4253-9d18-1063056486d5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:12.427640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:12.428328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:12.428616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:12.428904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:12.439727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:12.441250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:12.465560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1156/2359] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:22:14.611505 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:14.613090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:14.619692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:14.694588 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980161f-8e38-4ea9-ab4e-02e8b768d7d6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:14.701913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:14.702827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:14.703573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:14.703848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:14.705715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:14.757119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:14.759725 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:14.774276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:14.777762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:14.793296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1204/2360] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:14.859596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d11fb-6974-4f61-b58d-d0b859a44d37 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:14.869653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:14.870735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:14.871202 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:14.872147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:14.887709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:14.890022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:14.904299 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1157/2361] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:24.213799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:24.215307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:24.278318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a2852-3c9b-49d1-9fcf-66cb5e8c036c service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:24.282016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:24.282669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:24.282957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:24.283263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:24.285030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:24.296244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:24.297706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598d8a-34b9-40d5-8590-69f3f477c506 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:24.306338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1205/2362] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:24.898424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=57,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:26.536301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:26.613304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aebdc-ed31-465c-a876-d1b240fb7a0d service glance] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:26.618154 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:26.623527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:26.623527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:26.623527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] PATH_INFO: `/v3/users/5246e2b75484450881ec8c9056134536` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:26.623527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5246e2b75484450881ec8c9056134536)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:26.624067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:27.032421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5246e2b75484450881ec8c9056134536'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:27.033462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5246e2b75484450881ec8c9056134536'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:27.058853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5246e2b75484450881ec8c9056134536'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:27.075612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5246e2b75484450881ec8c9056134536'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:27.370500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5246e2b75484450881ec8c9056134536'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:27.384569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1158/2363] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:22:26 2022] DELETE /identity/v3/users/5246e2b75484450881ec8c9056134536 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:27.390372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:27.390852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:27.390963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:27.414659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:27.415966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:27.461987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:27.481633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1206/2364] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:22:27.492551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:27.494028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:27.561423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa58dd0-481d-44c6-8615-3b8154f74eb6 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:27.565196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:27.565868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:27.566158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:27.566441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:27.577482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:27.578972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:27.590653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1159/2365] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:28.417366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:28.418881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:28.432345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6bb83-8ee4-4e6d-bc37-c2e271931913 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:28.437396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:28.438301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:28.438583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:28.438865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] PATH_INFO: `/v3/projects/62facb800416476a9e42bd6e2bd03a18` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:28.440177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62facb800416476a9e42bd6e2bd03a18)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:28.442190 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:28.677217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62facb800416476a9e42bd6e2bd03a18', 'request_id': 'req-8f75f42b-4056-482e-89c4-47f1b418e371'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:28.683009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62facb800416476a9e42bd6e2bd03a18', 'request_id': 'req-8f75f42b-4056-482e-89c4-47f1b418e371'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:28.929739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62facb800416476a9e42bd6e2bd03a18', 'request_id': 'req-8f75f42b-4056-482e-89c4-47f1b418e371'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:28.944602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] The token cache is being invalidate because project 62facb800416476a9e42bd6e2bd03a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:22:28.944896 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:28.945760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8f75f42b-4056-482e-89c4-47f1b418e371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:22:28.947391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1207/2366] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:22:28 2022] DELETE /identity/v3/projects/62facb800416476a9e42bd6e2bd03a18 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:22:31.579094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:31.580774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:31.606143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddf26-6374-4218-93d7-4082c93030a9 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:31.630574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:31.669637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:31.670297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:31.670581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:31.670855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:31.678821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:31.689914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:31.691445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6440f25-a788-43ad-acc1-622dc49ec8a0 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:31.702358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1160/2367] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:32.267198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.268061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.268601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.271440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1208/2368] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:22:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:32.278369 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9714-5bea-4126-ae12-655fc50021f2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.278662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9714-5bea-4126-ae12-655fc50021f2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.278945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9714-5bea-4126-ae12-655fc50021f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.300321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0b6e9714-5bea-4126-ae12-655fc50021f2 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:32.301544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0b6e9714-5bea-4126-ae12-655fc50021f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:32.377123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1161/2369] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:22:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:22:32.391962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:32.396833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:32.482026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd543d-d1c2-461a-a6a7-f8785e7b268c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:32.485941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:32.488021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.488021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.488021 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.498801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:32.500307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7034a-ebcc-4bea-94a9-f92122e02fb2 service glance] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:32.512395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1209/2370] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:32.864859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.865778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.866005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.869697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1162/2371] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:22:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:32.874328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.874651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.874932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.895209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:32.896359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:32.910966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1210/2372] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:22:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:22:32.918979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:32.921311 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:32.936524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450352c1-3b55-4bce-a182-9274a03271ba None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:32.940776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:32.941922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:32.942321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:32.942704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:32.952250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:32.959418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6889cc2-4a20-40bc-8100-96737649b8f0 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:32.963374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1163/2373] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:22:33.039175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:33.040655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:33.053092 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e695cd-d212-4d17-a7cb-56649c01bf2b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:33.058252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:33.058448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:33.058787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:33.059696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:33.062005 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:33.070516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cb70-5d9f-4fa0-b02d-1811a457809b None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:33.070909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cb70-5d9f-4fa0-b02d-1811a457809b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:33.071403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cb70-5d9f-4fa0-b02d-1811a457809b None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:33.074051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1164/2374] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:22:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:22:33.075912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:33.077446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:33.083027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:33.083545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:33.084054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:33.086706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1211/2375] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:33.107384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:33.108694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:33.120136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1165/2376] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:22:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:22:33.133958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:33.145946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86cb2e-12f1-4d4f-927c-56a7c844743f service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:33.151869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:33.151869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:33.151869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:33.152455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:33.175248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:33.177257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:33.194206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1212/2377] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:33.282193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:33.283846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:33.376051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcfbc3-7ee6-4f97-8bf3-85b427309054 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:33.389145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:33.390050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:33.390491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:33.390938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:33.394199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:33.409028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:33.415595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda2e3c-fab3-40e2-aa23-9a2fb39c15a6 service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:33.426988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1166/2378] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:38.512808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:38.514912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:38.535273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a4b96-fe2b-48da-93d7-7181d299e221 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:38.541169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:38.541832 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:38.542113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:38.542945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:38.544862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:38.558795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:38.560571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:38.572461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1213/2379] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:22:40.031168 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f73c-13da-4a96-b013-6b18c4f8e9fe None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:40.032438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f73c-13da-4a96-b013-6b18c4f8e9fe None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:40.032878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f73c-13da-4a96-b013-6b18c4f8e9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:40.060368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7f06f73c-13da-4a96-b013-6b18c4f8e9fe None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:40.061231 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7f06f73c-13da-4a96-b013-6b18c4f8e9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:40.110373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:40.169283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1167/2380] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:22:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:22:40.205546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:40.207147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:40.293446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:40.345456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be852c8a-2301-4fc0-a8ff-07f7c3061ad9 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:40.350879 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:40.351660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:40.351947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:40.352227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:40.362061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:40.363977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:40.375234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1214/2381] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:22:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:22:50.328234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=13 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:51.529123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=21 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:59.490967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:59.491537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:59.491537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:59.509622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] MFA Rules not processed for user `ee2e478cdbb04617b48f388a4b38bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:59.510905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:59.571864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1168/2382] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:22:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:09.564648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=25 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:09.984519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:09.985988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:10.003192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4209b8-e5a1-4b27-85cc-5b96d7d161b8 service swift] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:10.011807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:10.011807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:10.011807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:10.011807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:10.012203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:10.025270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:10.026877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327197-bb81-4209-80b6-8add6ad2fb71 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:10.038241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1215/2383] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:23:17.494279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:17.509367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70010-fee7-405e-bb06-57c8ada45a7c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:17.513074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:17.513791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:17.514584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:17.515070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:17.517083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:17.595228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:17.596687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:17.606673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1169/2384] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:23:20.033512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:20.497111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.497857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.498498 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.524170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:20.525702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:20.533597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1216/2385] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:20.574655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.599323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df0df-17d4-4d9b-ba8e-2aaf471bab38 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.615738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.617267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.617741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.618177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.640568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.643499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.659668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1170/2386] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:23:20.699485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:20.727045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b40c-8a4d-48d6-94f9-d5cec3647457 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:20.740614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:20.741704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:20.743290 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:20.743813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:20.745833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:20.763595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:20.768714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e394f8-05f2-4844-b39a-2daebb561168 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:20.784929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1217/2387] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:23:30.652625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=36 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:30.774246 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=9 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:54.865430 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:54.881195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8303e-2fda-403a-be48-8fae6410c618 service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:54.889920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:54.891607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:54.891991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:54.892292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:54.894855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:54.992621 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:54.994574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:55.005967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1171/2388] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:24:05.001804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=28 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:34.337841 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919879-4863-452c-97c5-6b7906069d3a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:34.338442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919879-4863-452c-97c5-6b7906069d3a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:34.338442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919879-4863-452c-97c5-6b7906069d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:34.361775 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-89919879-4863-452c-97c5-6b7906069d3a None None] MFA Rules not processed for user `e3e8aa5fc932453e95080fa836dfac0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:34.375675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-89919879-4863-452c-97c5-6b7906069d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:34.375675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1218/2389] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:24:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:38.177807 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.190702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6413-7545-431c-85a8-ebacd59a1dd2 service nova] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.204719 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.205648 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.206040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.206040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.208232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:38.220928 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.222555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.234251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1172/2390] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:24:38.365111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89919879-4863-452c-97c5-6b7906069d3a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:38.367223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-89919879-4863-452c-97c5-6b7906069d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:38.441767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89919879-4863-452c-97c5-6b7906069d3a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:38.447606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665756-b968-44a3-8491-07594137c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:38.448494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665756-b968-44a3-8491-07594137c832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:38.448780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665756-b968-44a3-8491-07594137c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:38.449059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665756-b968-44a3-8491-07594137c832 admin admin] PATH_INFO: `/v3/users/ee2e478cdbb04617b48f388a4b38bd35` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:38.450243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665756-b968-44a3-8491-07594137c832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2e478cdbb04617b48f388a4b38bd35)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:38.452232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73665756-b968-44a3-8491-07594137c832 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:38.738031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-73665756-b968-44a3-8491-07594137c832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2e478cdbb04617b48f388a4b38bd35'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:38.747511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-73665756-b968-44a3-8491-07594137c832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2e478cdbb04617b48f388a4b38bd35'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:38.767524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-73665756-b968-44a3-8491-07594137c832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2e478cdbb04617b48f388a4b38bd35'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:38.769175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-73665756-b968-44a3-8491-07594137c832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2e478cdbb04617b48f388a4b38bd35'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:38.877022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-73665756-b968-44a3-8491-07594137c832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2e478cdbb04617b48f388a4b38bd35'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:38.886180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1219/2391] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:24:38 2022] DELETE /identity/v3/users/ee2e478cdbb04617b48f388a4b38bd35 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:39.270455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.282297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3035dc5-51f1-469a-85a7-b2d47ec176ad service placement] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.353714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.354615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.354914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.355264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] PATH_INFO: `/v3/projects/11215c9c5f59498c8110024722b1529c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.356544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11215c9c5f59498c8110024722b1529c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.358501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.525889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11215c9c5f59498c8110024722b1529c', 'request_id': 'req-6586b35f-22e1-4890-9c38-acd68c67da9f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:39.531987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11215c9c5f59498c8110024722b1529c', 'request_id': 'req-6586b35f-22e1-4890-9c38-acd68c67da9f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:39.546211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11215c9c5f59498c8110024722b1529c', 'request_id': 'req-6586b35f-22e1-4890-9c38-acd68c67da9f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:39.657268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] The token cache is being invalidate because project 11215c9c5f59498c8110024722b1529c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:24:39.657776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:39.658834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6586b35f-22e1-4890-9c38-acd68c67da9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:39.662215 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1173/2392] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:24:39 2022] DELETE /identity/v3/projects/11215c9c5f59498c8110024722b1529c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:24:39.667181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.668109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.668560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.687798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:39.689032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:39.756853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1220/2393] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:39.764779 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.765049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.765377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.787865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:39.788988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:39.797407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1174/2394] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:39.801769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.812549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa277aa-adeb-4687-b9cc-5b19e9912f7b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.820231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.820915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.821202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.821800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.822239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:24:39.822548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.824664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:39.834253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:39.841893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1221/2395] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:24:39.845381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:39.856252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218facb-8100-4f8e-938a-49a9f6fa4579 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:39.864174 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:39.864964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:39.865249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:39.865524 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:39.866214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:39.868244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.078488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1175/2396] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:24:39 2022] POST /identity/v3/projects => generated 399 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:40.083320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.093185 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09cfce-f693-43e1-bad2-1a2777bce051 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.152729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.153394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.153674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.153948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.154660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.156607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.326525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1222/2397] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:24:40 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:24:40.331204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.342549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43eb9-516c-421c-8b02-560bc353ceb4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.347275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.347985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.348273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.348548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.349320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.350915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.361689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1176/2398] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:24:40.365958 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.376555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27e06-4cb0-4b82-b5d3-3938a6c7625c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.382279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.382981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.383296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.383575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.384306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.385974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf53498-109f-4b6a-831a-e464f3c87796 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.396834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1223/2399] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:24:40.401264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.412079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02ec6f-8569-4739-991e-0dd6eef2f96f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.416772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.424265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.424265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.424265 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] PATH_INFO: `/v3/projects/07dcb0e7ee3348eb81b142691f681e1a/users/8fcd2f03148744cf8b7b0e260ae770d8/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.433061 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07dcb0e7ee3348eb81b142691f681e1a, user_id=8fcd2f03148744cf8b7b0e260ae770d8, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.435988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.611939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1177/2400] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:24:40 2022] PUT /identity/v3/projects/07dcb0e7ee3348eb81b142691f681e1a/users/8fcd2f03148744cf8b7b0e260ae770d8/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:40.623089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.623415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.623747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.651167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] MFA Rules not processed for user `8fcd2f03148744cf8b7b0e260ae770d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:40.652407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:40.720500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1224/2401] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:40.732001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:40.733749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:40.749269 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:40.793617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9821e-ba4c-440f-880c-6ddd555aa52d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:40.797372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:40.798065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:40.798347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:40.798619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:40.808405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:40.809854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343b758-f7f5-4b4b-bfbe-a5380581b839 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:40.821126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1178/2402] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:24:50.717580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:50.925089 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=20 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:52.532856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:52.667903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82141a-925b-449e-b612-fb3b8c23e9bf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:52.673156 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:52.674145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.674476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.674799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] PATH_INFO: `/v3/users/8fcd2f03148744cf8b7b0e260ae770d8` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.676754 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fcd2f03148744cf8b7b0e260ae770d8)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:52.678711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:52.932123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fcd2f03148744cf8b7b0e260ae770d8'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:52.939933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fcd2f03148744cf8b7b0e260ae770d8'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:52.954554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fcd2f03148744cf8b7b0e260ae770d8'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:52.955712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fcd2f03148744cf8b7b0e260ae770d8'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:53.037772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fcd2f03148744cf8b7b0e260ae770d8'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:53.048490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1225/2403] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:24:52 2022] DELETE /identity/v3/users/8fcd2f03148744cf8b7b0e260ae770d8 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:24:53.056101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.056680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.056823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.111882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:53.113352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:53.204521 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1179/2404] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:53.214831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.216463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:53.263098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:53.297343 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5094c-e841-41a3-92d8-58fad5d1b3d5 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.301090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.302326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.302721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.303047 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.325370 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.326852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.338844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1226/2405] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:24:53.805218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.806703 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:53.819971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab79ba-4508-4e4d-8f32-ba1af16711b2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.825262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.826092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.826382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.826664 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] PATH_INFO: `/v3/projects/07dcb0e7ee3348eb81b142691f681e1a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.827916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07dcb0e7ee3348eb81b142691f681e1a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.830100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.950493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07dcb0e7ee3348eb81b142691f681e1a', 'request_id': 'req-b7f64355-3249-402c-8cb8-99daa40a3c71'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:53.956331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07dcb0e7ee3348eb81b142691f681e1a', 'request_id': 'req-b7f64355-3249-402c-8cb8-99daa40a3c71'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:53.963309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07dcb0e7ee3348eb81b142691f681e1a', 'request_id': 'req-b7f64355-3249-402c-8cb8-99daa40a3c71'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:54.051099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] The token cache is being invalidate because project 07dcb0e7ee3348eb81b142691f681e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:24:54.051500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:54.052377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:24:54.053948 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1180/2406] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:24:53 2022] DELETE /identity/v3/projects/07dcb0e7ee3348eb81b142691f681e1a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:03.382829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=16 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:04.047742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:06.715302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:06.716720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:06.825461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94700b9-6d00-4740-9d74-cb8a68450d3d service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:06.894975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:06.895857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:06.896017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:06.896275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:06.899302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:06.911734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:06.914607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:06.925956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1227/2407] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:11.144392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:11.145959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:11.268733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f64355-3249-402c-8cb8-99daa40a3c71 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:11.275242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:11.275932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:11.276255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:11.276559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:11.278486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:11.346600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:11.348270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e74235-2435-4571-ae0c-62a33af280cb service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:11.359187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1181/2408] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:25:16.920942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=67 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:21.354726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=68 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:23.123726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:23.125115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:23.142763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efd3a-e120-4200-9ae1-a39b9fc19768 service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:23.148334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:23.149607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:23.150522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:23.150843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:23.152788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:23.224603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:23.226369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bf7d2-5ef1-416b-963e-3c7a3476765c service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:23.236755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1228/2409] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:24.513176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:24.513781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:24.513781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:24.539367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] MFA Rules not processed for user `3ac486b2d0ac42f9ba2fed7b17418b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:24.540967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:24.604407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1182/2410] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:25:33.231999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=28 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:34.598210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=25 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:34.741018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:34.752734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:34.752734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:34.773610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:34.775441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:34.841053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1229/2411] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:25:35.001421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:35.005438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:35.129151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936f99-9720-4bb9-a0db-4635126d9118 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:35.136661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:35.137268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:35.137618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:35.137884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:35.147851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:35.149478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:35.160121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1183/2412] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:25:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:25:36.493041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:36.495181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:36.578223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c88f7-3d0f-4370-ad6e-f1228c069763 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:36.582101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:36.588671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:36.588671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:36.588671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:36.588671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:36.651666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:36.653119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:36.662917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1230/2413] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:25:40.235978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:40.237855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:40.255698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde1143-77e3-4742-bcc2-9c2dd9c87f07 service swift] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:40.273578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:40.274574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:40.275706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:40.276241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:40.279499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:40.306544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:40.309062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:40.322912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1184/2414] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:44.899726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:44.902071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:44.915200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c21f0-e150-4f39-9248-6e233ed82b47 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:44.924811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:44.932170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:44.932170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:44.932170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] PATH_INFO: `/v3/users/41bc56e0e3d0451482572959451ae043` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:44.932170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bc56e0e3d0451482572959451ae043)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:44.932170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:44.955931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:45.162239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bc56e0e3d0451482572959451ae043'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.171561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bc56e0e3d0451482572959451ae043'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.192787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bc56e0e3d0451482572959451ae043'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.193869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bc56e0e3d0451482572959451ae043'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.278646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bc56e0e3d0451482572959451ae043'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.285176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1231/2415] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:25:44 2022] DELETE /identity/v3/users/41bc56e0e3d0451482572959451ae043 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:45.577151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:45.593726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c1bf-361d-49fe-a5c9-ec93cab8c224 service placement] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:45.669801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:45.673546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:45.674359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:45.674665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] PATH_INFO: `/v3/projects/20b87d0f8d1c4ab8a9218ec96872750f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:45.675998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b87d0f8d1c4ab8a9218ec96872750f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:45.677951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:45.817395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b87d0f8d1c4ab8a9218ec96872750f', 'request_id': 'req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.824092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b87d0f8d1c4ab8a9218ec96872750f', 'request_id': 'req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.830872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b87d0f8d1c4ab8a9218ec96872750f', 'request_id': 'req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.919038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] The token cache is being invalidate because project 20b87d0f8d1c4ab8a9218ec96872750f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:25:45.919787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.920463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:45.922078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1185/2416] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:25:45 2022] DELETE /identity/v3/projects/20b87d0f8d1c4ab8a9218ec96872750f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:45.928146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:45.929625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:45.944069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6654-5011-4545-9ecc-64dd14b8e12a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:46.024504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:46.026589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:46.026589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:46.027045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] PATH_INFO: `/v3/users/99463be973b5492297d1a291323a77be` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:46.028725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99463be973b5492297d1a291323a77be)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:46.031297 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:46.235287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99463be973b5492297d1a291323a77be'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.243098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99463be973b5492297d1a291323a77be'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.262454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99463be973b5492297d1a291323a77be'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.262454 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99463be973b5492297d1a291323a77be'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.356225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-319efc80-b58a-4857-b383-b718d1cebe6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99463be973b5492297d1a291323a77be'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.364982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1232/2417] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:25:45 2022] DELETE /identity/v3/users/99463be973b5492297d1a291323a77be => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:46.677490 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:46.684079 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:46.694705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075fabb-a7bc-44d9-9ad5-6b09ff3afb41 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:46.699412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:46.759015 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:46.760491 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:46.760776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:46.761052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] PATH_INFO: `/v3/projects/c5fabcb3cf23446192558faa11c75e60` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:46.762252 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fabcb3cf23446192558faa11c75e60)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:46.764212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:46.940645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fabcb3cf23446192558faa11c75e60', 'request_id': 'req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.950830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fabcb3cf23446192558faa11c75e60', 'request_id': 'req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:46.957788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fabcb3cf23446192558faa11c75e60', 'request_id': 'req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:47.047140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] The token cache is being invalidate because project c5fabcb3cf23446192558faa11c75e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:25:47.047543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:47.048289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2b3078f8-6bed-42ba-97d9-08e8e3d0b4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:25:47.050257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1186/2418] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:25:46 2022] DELETE /identity/v3/projects/c5fabcb3cf23446192558faa11c75e60 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:47.055742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.057417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.057907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.083438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:47.114103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:47.115352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:47.188453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1233/2419] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:25:47.198580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.198877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.199232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.218987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:47.220327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:47.229794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1187/2420] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:25:47.235201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.251885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe9a11-197b-4961-b856-8ddac92a1289 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.258529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.258529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.258529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.258529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.259576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:25:47.259906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.261631 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.275632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1234/2421] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:25:47.280533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.292870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfdf9-4ee5-47f1-ad87-de8407d2d445 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.297292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.297962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.298582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.298582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.299230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.301233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.397342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1188/2422] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:25:47 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:47.399471 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.417723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b58cf-6ccf-4015-a42f-b399d7354712 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.511850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.512597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.512882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.519034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.520271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.522899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.620163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1235/2423] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:25:47 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:47.624396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.639520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b15b-d825-4d15-a009-95f8f5ef4c4d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.646229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.646670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.647138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.647431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.648321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.649838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.662878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1189/2424] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:25:47.667101 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.679709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb334-2b95-43a3-9857-db596774bc61 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.686729 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.687444 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.687891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.688082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.688762 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.690316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730427-c5bc-40ac-8456-49a559fe74d0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.702386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1236/2425] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:25:47.706597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:47.718937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a926e9-9c96-418f-a4c3-c4d34a04ce02 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:47.724198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:47.725808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.725808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.725808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] PATH_INFO: `/v3/projects/f3904fb55e2e463fb548333d97601ce5/users/c466157931bd41baabb01abbec492b4f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.750008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3904fb55e2e463fb548333d97601ce5, user_id=c466157931bd41baabb01abbec492b4f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:47.753273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:47.882484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1190/2426] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:25:47 2022] PUT /identity/v3/projects/f3904fb55e2e463fb548333d97601ce5/users/c466157931bd41baabb01abbec492b4f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:47.889192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2577e0-8fa2-441e-a00e-c126d425538a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:47.889637 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2577e0-8fa2-441e-a00e-c126d425538a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:47.889990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2577e0-8fa2-441e-a00e-c126d425538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:47.909549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8a2577e0-8fa2-441e-a00e-c126d425538a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:47.910887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8a2577e0-8fa2-441e-a00e-c126d425538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:47.983778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:47.994375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1237/2427] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:25:48.005081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:48.006899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:48.054271 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:48.081347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4833-995a-4619-b38e-cf8cffcfc070 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:48.085707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:48.086374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:48.086661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:48.086943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:48.097498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:48.098972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:48.109788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1191/2428] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:54.484524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:54.485076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:54.485177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:54.523524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] MFA Rules not processed for user `c466157931bd41baabb01abbec492b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:25:54.526233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:54.664922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1238/2429] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:25:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:25:54.679795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:54.692238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef4416-0d8f-44c0-a086-92fd01ac5a14 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:54.699353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:54.700358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:54.700639 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:54.700918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:54.701795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:54.703797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:54.825462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1192/2430] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:25:54 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:25:54.831465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:54.840756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0688f-2191-403c-ae59-03847f3d7d40 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:54.908237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:54.908921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:54.909202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:54.909477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:54.910176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:54.912150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:55.014540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1239/2431] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:25:54 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:25:55.019835 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:55.033709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f752989-c39e-496b-8dc3-243b15ec7b64 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:55.039560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:55.040251 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:55.041118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:55.041417 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:55.042156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:55.043892 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:55.055528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1193/2432] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:25:55.061023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:55.073269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4f1f4-ff36-4661-9fbb-df98cd03be94 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:55.078369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:55.079880 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:55.080350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:55.080816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] PATH_INFO: `/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d/users/e29df8c17adb49a28acec3d796dfb1c1/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:55.098225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6e12d349c24bd885349dbd2e4cc91d, user_id=e29df8c17adb49a28acec3d796dfb1c1, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:55.100936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:55.190325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1240/2433] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:25:55 2022] PUT /identity/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d/users/e29df8c17adb49a28acec3d796dfb1c1/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:25:55.194967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:55.208864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a1e87-50cd-41a9-a29d-83f9fd3c6fd8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:55.276833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:55.277458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:55.277744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:55.278021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:55.278789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:55.280468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e3407-0b34-4e84-83c9-e05994f15158 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:55.298219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1194/2434] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:25:55.299506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:55.310996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9e8b-5c37-4db8-8e89-09d8c70dcfbb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:55.316391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:55.317295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:55.317604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:55.317888 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] PATH_INFO: `/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d/users/e29df8c17adb49a28acec3d796dfb1c1/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:55.320031 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6e12d349c24bd885349dbd2e4cc91d, user_id=e29df8c17adb49a28acec3d796dfb1c1, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:55.323360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:55.334400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1241/2435] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:25:55 2022] PUT /identity/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d/users/e29df8c17adb49a28acec3d796dfb1c1/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:03.129019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:03.129630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:03.129630 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:03.160798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] MFA Rules not processed for user `e29df8c17adb49a28acec3d796dfb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:03.162044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:03.249726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1195/2436] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:26:03.264831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:03.266766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:03.320332 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:03.353367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fe251-6557-4a09-91a2-0926c3ecb418 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:03.360129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:03.360797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:03.361121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:03.361569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:03.426315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:03.427873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:03.438074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1242/2437] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:11.696665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:11.726274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:11.760224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e3b70-7cc7-46cd-b988-94299ec70e6f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:11.764468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:11.765771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:11.766142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:11.766353 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:11.769262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:11.863548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:11.866722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:11.882135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1196/2438] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:13.433368 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=37 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:16.137164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:16.142334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:16.184534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6da253-78c0-4c02-a259-ef1944ba8075 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:16.191500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:16.194562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:16.194987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:16.195361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] PATH_INFO: `/v3/users/e3e8aa5fc932453e95080fa836dfac0c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:16.197477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e8aa5fc932453e95080fa836dfac0c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:16.200583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:16.516367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e8aa5fc932453e95080fa836dfac0c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:16.524127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e8aa5fc932453e95080fa836dfac0c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:16.540009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e8aa5fc932453e95080fa836dfac0c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:16.541287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e8aa5fc932453e95080fa836dfac0c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:16.607711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-61f69851-e782-4ea0-997e-0dee5d70e6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e8aa5fc932453e95080fa836dfac0c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:16.614865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1243/2439] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:16 2022] DELETE /identity/v3/users/e3e8aa5fc932453e95080fa836dfac0c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:16.908394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:16.921890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d1e25-02fc-4e69-af79-02dd5f667be8 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:16.988949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:16.989832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:16.990123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:16.990405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] PATH_INFO: `/v3/projects/08cc50e4caea44e2a2bb1413a35e3b6d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:16.991641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cc50e4caea44e2a2bb1413a35e3b6d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:16.993746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:17.173730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cc50e4caea44e2a2bb1413a35e3b6d', 'request_id': 'req-beda92b1-f166-440b-ad1e-b1f7fc30f74f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:17.181218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cc50e4caea44e2a2bb1413a35e3b6d', 'request_id': 'req-beda92b1-f166-440b-ad1e-b1f7fc30f74f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:17.192566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cc50e4caea44e2a2bb1413a35e3b6d', 'request_id': 'req-beda92b1-f166-440b-ad1e-b1f7fc30f74f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:17.302187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] The token cache is being invalidate because project 08cc50e4caea44e2a2bb1413a35e3b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:17.302481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:17.303153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-beda92b1-f166-440b-ad1e-b1f7fc30f74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:17.304715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1197/2440] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:16 2022] DELETE /identity/v3/projects/08cc50e4caea44e2a2bb1413a35e3b6d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:17.311430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.317703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.317703 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.337795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:17.339039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:17.407899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1244/2441] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:17.419191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.419484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.419765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.427753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:17.440112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:17.444886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:17.450107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1198/2442] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:17.454429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:17.465652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6394d-c616-454f-9f46-90bc8710477c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:17.470627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:17.471669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.472317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.472713 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.474074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:17.474537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:17.476996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:17.491056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1245/2443] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:17.495877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:17.506029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb360-3324-410e-ab33-f52820d1392a None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:17.512184 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:17.512853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.513138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.513418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.514112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:17.516144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:17.695988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1199/2444] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:17 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:17.701283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:17.712181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5ff4-bf48-458b-9711-8a1fe31d2515 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:17.784549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:17.785227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.785508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.785779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.786470 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:17.788481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:17.796062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:17.939763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1246/2445] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:17 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:17.943922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:17.956829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9ec95-bf93-41ca-acac-8c8a016d0ccf admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:17.963235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:17.963972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:17.964261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:17.964544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:17.965291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:17.967262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:17.979850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1200/2446] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:17.986255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:17.997722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464f256-619e-4b80-89c6-f6e7a8a693f5 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:18.003384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:18.004066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:18.004354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:18.004639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:18.005374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:18.007015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43714d9b-5ce7-4a83-b6bf-f9e0c948157a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:18.020330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1247/2447] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:18.028197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:18.039750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb898b9-0d4c-4bc4-a106-7e728d856cdd admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:18.045134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:18.046091 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:18.046382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:18.046658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] PATH_INFO: `/v3/projects/027a4768ba6a4d3f8967481442622a2d/users/33a5edf467d340338cea978e2f86eb75/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:18.063148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=027a4768ba6a4d3f8967481442622a2d, user_id=33a5edf467d340338cea978e2f86eb75, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:18.066038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:18.214547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1201/2448] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:18 2022] PUT /identity/v3/projects/027a4768ba6a4d3f8967481442622a2d/users/33a5edf467d340338cea978e2f86eb75/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:18.225558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:18.226078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:18.232504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:18.256175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] MFA Rules not processed for user `33a5edf467d340338cea978e2f86eb75`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:18.257496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:18.334216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1248/2449] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:18.346993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:18.350864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:18.439074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b21d-dd1c-4518-ac9d-08106708f13a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:18.444799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:18.445469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:18.445751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:18.446039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:18.455151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:18.456771 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0ec2b-b25f-406c-996e-65b7f2e93c64 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:18.467196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1202/2450] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:23.023565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:23.096345 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c473833-b93e-4637-9f16-6ea7f5e51e5c None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:23.101106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:23.102008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:23.102298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:23.102594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] PATH_INFO: `/v3/users/33a5edf467d340338cea978e2f86eb75` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:23.104511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a5edf467d340338cea978e2f86eb75)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:23.106478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:23.450748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a5edf467d340338cea978e2f86eb75'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:23.458116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a5edf467d340338cea978e2f86eb75'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:23.469633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:23.472380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a5edf467d340338cea978e2f86eb75'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:23.473232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a5edf467d340338cea978e2f86eb75'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:23.647714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a5edf467d340338cea978e2f86eb75'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:23.656122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1249/2451] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:23 2022] DELETE /identity/v3/users/33a5edf467d340338cea978e2f86eb75 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:23.660980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:23.661548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:23.661548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:23.682436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:23.696211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:23.751578 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1203/2452] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:26:23.762426 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:23.763959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:23.828308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7f7c1-2cba-4993-99f1-a875ad0fe9b3 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:23.832053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:23.832716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:23.832995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:23.833269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:23.844553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:23.845979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ecbcd-60e4-466c-9687-84410980de30 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:23.857040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1250/2453] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:24.409163 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:24.411347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:24.425675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d26d3b-7cf7-4796-b3fa-72c18f0fd8a5 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:24.432556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:24.433642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:24.433954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:24.434261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] PATH_INFO: `/v3/projects/027a4768ba6a4d3f8967481442622a2d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:24.435565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=027a4768ba6a4d3f8967481442622a2d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:24.438123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:24.469100 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:24.628697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027a4768ba6a4d3f8967481442622a2d', 'request_id': 'req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:24.634607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027a4768ba6a4d3f8967481442622a2d', 'request_id': 'req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:24.641085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027a4768ba6a4d3f8967481442622a2d', 'request_id': 'req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:24.825997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] The token cache is being invalidate because project 027a4768ba6a4d3f8967481442622a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:24.826691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:24.827219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-2475dea9-2f44-4509-8ca5-cb1404ad9cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:24.829180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1204/2454] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:24 2022] DELETE /identity/v3/projects/027a4768ba6a4d3f8967481442622a2d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:26:24.834282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:24.834930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:24.834930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:24.852914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:24.854121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:24.916367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1251/2455] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:24.924126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:24.924428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:24.924711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:24.943765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:24.944962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:24.953732 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1205/2456] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:24.958070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:24.968524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172598-3bc4-4818-80b2-d86f8d7ab063 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:24.974786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:24.975486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:24.975812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:24.976113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:24.976872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:24.977198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:24.978919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:24.991443 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1252/2457] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:24.996716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.006248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c8baf-8bf6-4b3d-ac1b-5889e748fd1e None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.012142 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.012794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.013073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.013344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.014055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.016003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:25.107167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1206/2458] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:24 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:25.111060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.122195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cceb7d-c161-4b1f-ada2-f9a547a5405a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.179753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.180538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.180893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.181245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.182852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.184939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:25.333798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1253/2459] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:25 2022] POST /identity/v3/users => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:26:25.338232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.349894 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc1ad-193f-49a2-b42f-288d7997ccb8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.355742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.360230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.360230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.360230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.360230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.360230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:25.372922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1207/2460] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:25.377566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.389468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d77c33-ae12-4cb3-b01b-06aa2c6cb84e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.394328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.395075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.395395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.395676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.396414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.398104 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82334a0-e398-4646-9212-8769ecefc8ee admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:25.399440 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:25.410615 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1254/2461] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:25.415095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.433185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d4f5-a4e2-4917-b554-cf9625858772 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.437832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.438755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.439039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.439350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] PATH_INFO: `/v3/projects/35a9dd63969c4b43997df0e61bf5a9a3/users/75b29da1c47e4777be119faab346d606/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.453960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a9dd63969c4b43997df0e61bf5a9a3, user_id=75b29da1c47e4777be119faab346d606, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.456750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:25.636885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1208/2462] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:25 2022] PUT /identity/v3/projects/35a9dd63969c4b43997df0e61bf5a9a3/users/75b29da1c47e4777be119faab346d606/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:25.649553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.649848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.650129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.678728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] MFA Rules not processed for user `75b29da1c47e4777be119faab346d606`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:25.680330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:25.745137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1255/2463] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:25.750155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:25.812660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68135e4a-dec9-41d0-8455-f03ffb301a97 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:25.817258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:25.817920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:25.818203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:25.818502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:25.819232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:25.821127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.066157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1209/2464] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:25 2022] POST /identity/v3/projects => generated 405 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:26.070302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.081954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad58a3-a778-4cda-b5ca-bd5535b426d0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:26.140413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:26.141354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.141643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.141920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.142608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:26.144565 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.362595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1256/2465] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:26 2022] POST /identity/v3/users => generated 309 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:26.366744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.378565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee327d3-52bf-4cdd-a4b4-92b00547375c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:26.383722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:26.384399 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.389217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.389217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.389217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:26.389217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.399318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1210/2466] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:26.403640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.413936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187aac4-69b5-41be-9565-e1a1fae2f7c7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:26.418580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:26.419486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.419769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.420045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] PATH_INFO: `/v3/projects/b36c328904454b51aa85eea529cd8cc6/users/dbf6fa880e7b465d83bd9a9f7187449f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.436028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36c328904454b51aa85eea529cd8cc6, user_id=dbf6fa880e7b465d83bd9a9f7187449f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:26.439000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.663802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1257/2467] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:26 2022] PUT /identity/v3/projects/b36c328904454b51aa85eea529cd8cc6/users/dbf6fa880e7b465d83bd9a9f7187449f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:26.668423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.681264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891750b3-e9db-4a87-8e89-031f4547f8e3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:26.699684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:26.740546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:26.741254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.741541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.741820 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.742552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:26.744275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c29b04-708b-4ed5-9469-0e25d0e7fc40 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.755843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1211/2468] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:26.760114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.772238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1c97-4472-4f85-a266-0a2300e7915d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:26.777890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:26.778764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.779042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.779342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] PATH_INFO: `/v3/projects/b36c328904454b51aa85eea529cd8cc6/users/dbf6fa880e7b465d83bd9a9f7187449f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.781285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36c328904454b51aa85eea529cd8cc6, user_id=dbf6fa880e7b465d83bd9a9f7187449f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:26.784011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:26.794906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1258/2469] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:26 2022] PUT /identity/v3/projects/b36c328904454b51aa85eea529cd8cc6/users/dbf6fa880e7b465d83bd9a9f7187449f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:26.805040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:26.805331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:26.805610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:26.832800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] MFA Rules not processed for user `dbf6fa880e7b465d83bd9a9f7187449f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:26.834016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:26.908661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1212/2470] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:26.919960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:26.921561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:26.966419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:27.002144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397c394-5337-4d0c-a4f5-ec142591aaff admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:27.005926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:27.006573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:27.006859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:27.007161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:27.018560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:27.020099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26702b2b-72da-4bf3-a7d7-9fc2da9c116b service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:27.030433 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1259/2471] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:26 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:27.194191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:27.257264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dad4f-b518-4af3-9b8b-f533f7353fb1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:27.262338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:27.263242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:27.263525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:27.263801 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] PATH_INFO: `/v3/users/75b29da1c47e4777be119faab346d606` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:27.264979 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b29da1c47e4777be119faab346d606)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:27.267012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:27.679257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b29da1c47e4777be119faab346d606'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:27.679910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b29da1c47e4777be119faab346d606'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:27.691927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b29da1c47e4777be119faab346d606'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:27.698705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b29da1c47e4777be119faab346d606'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:27.840090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b29da1c47e4777be119faab346d606'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:27.846588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1213/2472] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:27 2022] DELETE /identity/v3/users/75b29da1c47e4777be119faab346d606 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:27.851218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:27.851711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:27.851815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:27.872406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:27.873695 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:27.943728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1260/2473] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:27.954412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:27.955819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:28.001765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:28.017860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbffbf-708f-4e9c-9e69-3c264dd9663f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:28.021537 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:28.022192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:28.022471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:28.022746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:28.032576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:28.034192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:28.044487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1214/2474] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:28.563640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:28.565312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:28.582316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4d2aa-1fc1-457b-8d3c-20b1909a2962 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:28.585417 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:28.586295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:28.586583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:28.586859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] PATH_INFO: `/v3/projects/35a9dd63969c4b43997df0e61bf5a9a3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:28.588866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a9dd63969c4b43997df0e61bf5a9a3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:28.591760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:28.863847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a9dd63969c4b43997df0e61bf5a9a3', 'request_id': 'req-cc025baa-d675-45f4-a913-891892b89253'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:28.869658 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a9dd63969c4b43997df0e61bf5a9a3', 'request_id': 'req-cc025baa-d675-45f4-a913-891892b89253'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.091299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a9dd63969c4b43997df0e61bf5a9a3', 'request_id': 'req-cc025baa-d675-45f4-a913-891892b89253'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.103232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] The token cache is being invalidate because project 35a9dd63969c4b43997df0e61bf5a9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:29.103523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.104158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.105735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1261/2475] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:28 2022] DELETE /identity/v3/projects/35a9dd63969c4b43997df0e61bf5a9a3 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:29.109967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:29.120442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54457e5-7e49-4033-ab0c-a83381752e8e service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:29.193114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:29.194009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:29.194318 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:29.194592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] PATH_INFO: `/v3/users/dbf6fa880e7b465d83bd9a9f7187449f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:29.195969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf6fa880e7b465d83bd9a9f7187449f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:29.198734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:29.602667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf6fa880e7b465d83bd9a9f7187449f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.603355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf6fa880e7b465d83bd9a9f7187449f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.617041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf6fa880e7b465d83bd9a9f7187449f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.619214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:29.629730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf6fa880e7b465d83bd9a9f7187449f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.790595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-214e71d4-3beb-48e8-b5e5-7e35650a545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf6fa880e7b465d83bd9a9f7187449f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:29.798151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1215/2476] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:29 2022] DELETE /identity/v3/users/dbf6fa880e7b465d83bd9a9f7187449f => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:30.444607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:30.446822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:30.468980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025baa-d675-45f4-a913-891892b89253 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:30.526997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:30.547728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:30.548569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:30.548852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:30.549129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] PATH_INFO: `/v3/projects/b36c328904454b51aa85eea529cd8cc6` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:30.550524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36c328904454b51aa85eea529cd8cc6)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:30.552496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:30.778780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36c328904454b51aa85eea529cd8cc6', 'request_id': 'req-5f385138-a5db-4f2d-ad21-085de338816b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:30.784253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36c328904454b51aa85eea529cd8cc6', 'request_id': 'req-5f385138-a5db-4f2d-ad21-085de338816b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:30.967938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36c328904454b51aa85eea529cd8cc6', 'request_id': 'req-5f385138-a5db-4f2d-ad21-085de338816b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:30.981023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] The token cache is being invalidate because project b36c328904454b51aa85eea529cd8cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:30.981314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:30.981974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-5f385138-a5db-4f2d-ad21-085de338816b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:30.983603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1262/2477] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:30 2022] DELETE /identity/v3/projects/b36c328904454b51aa85eea529cd8cc6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:30.988607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:30.989097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.004117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.008370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:31.009651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:31.069655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1216/2478] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:31.078373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.078758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.079041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.098176 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:31.099434 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:31.107462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1263/2479] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:31.112096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.123062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac1867-48b7-4424-b8ac-badb2fba1e7f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.127914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.128582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.128871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.129152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.129897 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:31.130200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.131861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.150496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1217/2480] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:26:31.167040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.176497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93eaa9-011c-4f62-bd83-c2d2f8827e2a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.181287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.181950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.182373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.182906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.183416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.185414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.415181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1264/2481] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:31 2022] POST /identity/v3/projects => generated 357 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:31.420118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.430173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938beb3c-0ad6-4af6-933a-ce52da38ca60 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.493544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.494241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.494708 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.495033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.495617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.497788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.539018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.540756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:31.616822 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f367b-0d24-49b5-b1d5-47e4e2f906e1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.622629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.623334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.623618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.623893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.625629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:31.648385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1218/2482] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:31 2022] POST /identity/v3/users => generated 279 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:31.653382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.663699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60bbb-3970-4e83-8a59-119620809fe0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.669200 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.669809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.670234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.670518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.671350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.672981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.684307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1219/2483] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:31.688825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.692214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.694152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.698956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a488-0305-40a2-af8f-abf6169fa723 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.704128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.704594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1265/2484] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:31.704846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.705134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.705413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.706285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.708153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2c1e-33c1-42ad-842e-510dce0b8377 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.721455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1220/2485] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:26:31.725523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:31.737117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34ae17-239f-4c2f-96a2-1b0d47e3f1af service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:31.742108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:31.743017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.743328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.743821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] PATH_INFO: `/v3/projects/faec025a9e534c1e80402bf36fcf834d/users/831a457a6e814fdb8aa1d60265bde379/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.760949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faec025a9e534c1e80402bf36fcf834d, user_id=831a457a6e814fdb8aa1d60265bde379, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:31.763806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:31.943064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1266/2486] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:31 2022] PUT /identity/v3/projects/faec025a9e534c1e80402bf36fcf834d/users/831a457a6e814fdb8aa1d60265bde379/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:31.955701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:31.955980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:31.956233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:31.983884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:31.992115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] MFA Rules not processed for user `831a457a6e814fdb8aa1d60265bde379`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:32.002828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:32.021769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:32.065834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1221/2487] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:32.070302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.134167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a728d57-de11-4fd6-b8fd-eb8abe3cd757 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.139843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.140566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.140788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.141066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.141779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.143751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:32.295501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1267/2488] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:32 2022] POST /identity/v3/projects => generated 355 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:32.299609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.311139 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced0e7c-27a6-4b5f-808f-2c1d789507eb None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.372127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.372798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.373081 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.373358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.374038 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.376103 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:32.568587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1222/2489] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:32 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:32.572653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.583972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98eb6-9b34-4821-a899-564da0a8238a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.588749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.589428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.593221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.593221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.593221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.593221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:32.604751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1268/2490] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:32.609050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.619515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed85e2-45ef-4034-a245-c364686b672e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.626356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.627286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.627569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.627843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] PATH_INFO: `/v3/projects/0fbba9e783384fc8953435e5cb3b07db/users/16c5c900840a4f228792ebd2f32943b0/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.644487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbba9e783384fc8953435e5cb3b07db, user_id=16c5c900840a4f228792ebd2f32943b0, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.647915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:32.864937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1223/2491] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:32 2022] PUT /identity/v3/projects/0fbba9e783384fc8953435e5cb3b07db/users/16c5c900840a4f228792ebd2f32943b0/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:32.869247 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.881873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb49e5c-ddf1-465a-a37e-163cdc4365ca admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.949324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.950019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.950298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.950573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.951365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.952981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29418857-8edb-4ec0-97e9-2346285a8850 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:32.964054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1269/2492] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:32.968877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:32.980785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa03fb-831b-497e-a977-48f95853927f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:32.987772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:32.988679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:32.988968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:32.989255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] PATH_INFO: `/v3/projects/0fbba9e783384fc8953435e5cb3b07db/users/16c5c900840a4f228792ebd2f32943b0/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:32.991307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbba9e783384fc8953435e5cb3b07db, user_id=16c5c900840a4f228792ebd2f32943b0, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:32.993978 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:33.006525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1224/2493] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:32 2022] PUT /identity/v3/projects/0fbba9e783384fc8953435e5cb3b07db/users/16c5c900840a4f228792ebd2f32943b0/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:33.017150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:33.017150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:33.017597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:33.049675 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] MFA Rules not processed for user `16c5c900840a4f228792ebd2f32943b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:33.050912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:33.072828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:33.126457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1270/2494] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:26:33.137992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:33.139781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:33.205300 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c16b59-8e82-40d0-ad34-eabd1296482f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:33.217123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:33.217794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:33.218087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:33.218371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:33.229744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:33.231753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336756-8edd-4e54-8c03-b4e4149dcbb4 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:33.242010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1225/2495] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:33.467060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:33.479744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23236cd8-d185-445b-ad94-be5af6de5815 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:33.483769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:33.484398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:33.484789 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:33.484997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:33.548422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:33.550172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:33.561801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1271/2496] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:34.187546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f4fc-864e-4523-b244-6e73a5da402a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:34.188528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f4fc-864e-4523-b244-6e73a5da402a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:34.189096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39f4fc-864e-4523-b244-6e73a5da402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:34.209893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-3e39f4fc-864e-4523-b244-6e73a5da402a None None] MFA Rules not processed for user `c466157931bd41baabb01abbec492b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:34.211326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3e39f4fc-864e-4523-b244-6e73a5da402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:34.232074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:34.394700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1226/2497] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:35.252791 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:35.320997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a354ce0-911b-4adc-8ee6-3fc02d227757 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:35.326869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:35.327786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:35.328070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:35.328349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] PATH_INFO: `/v3/users/831a457a6e814fdb8aa1d60265bde379` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:35.329616 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=831a457a6e814fdb8aa1d60265bde379)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:35.331600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:35.509508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831a457a6e814fdb8aa1d60265bde379'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:35.516295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:35.518466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831a457a6e814fdb8aa1d60265bde379'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:35.531267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831a457a6e814fdb8aa1d60265bde379'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:35.532095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831a457a6e814fdb8aa1d60265bde379'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:35.611115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831a457a6e814fdb8aa1d60265bde379'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:35.624614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1272/2498] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:35 2022] DELETE /identity/v3/users/831a457a6e814fdb8aa1d60265bde379 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:26:35.626370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:35.626976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:35.626976 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:35.662135 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:35.663618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:35.734151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1227/2499] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:35.744292 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:35.745878 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:35.810121 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e70f-7a51-4662-a06a-ccd331fcc91e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:35.814259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:35.814843 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:35.815148 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:35.815431 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:35.826325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:35.827920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:35.837960 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1273/2500] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:36.266634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:36.268101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:36.280533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51a71-e67c-4166-99a1-611c3ff8c80b None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:36.285255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:36.288028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:36.288028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:36.288028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] PATH_INFO: `/v3/projects/faec025a9e534c1e80402bf36fcf834d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:36.288028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faec025a9e534c1e80402bf36fcf834d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:36.289867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:36.468172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faec025a9e534c1e80402bf36fcf834d', 'request_id': 'req-25587bd1-ec62-4f84-878e-2ca771c69458'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:36.474064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faec025a9e534c1e80402bf36fcf834d', 'request_id': 'req-25587bd1-ec62-4f84-878e-2ca771c69458'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:36.481096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faec025a9e534c1e80402bf36fcf834d', 'request_id': 'req-25587bd1-ec62-4f84-878e-2ca771c69458'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:36.596294 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] The token cache is being invalidate because project faec025a9e534c1e80402bf36fcf834d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:36.596593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:36.597226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:36.598797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1228/2501] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:36 2022] DELETE /identity/v3/projects/faec025a9e534c1e80402bf36fcf834d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:36.602903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:36.613662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616a5e-1b94-4471-a946-f189719062fe service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:36.677547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:36.678450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:36.678724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:36.678992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] PATH_INFO: `/v3/users/16c5c900840a4f228792ebd2f32943b0` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:36.680234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c5c900840a4f228792ebd2f32943b0)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:36.682357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:37.040684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c5c900840a4f228792ebd2f32943b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.049270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c5c900840a4f228792ebd2f32943b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.063335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c5c900840a4f228792ebd2f32943b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.064075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c5c900840a4f228792ebd2f32943b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.066668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:37.143551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-82ed575f-a156-4a54-9c5c-0d6c8bb33a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c5c900840a4f228792ebd2f32943b0'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.149973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1274/2502] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:36 2022] DELETE /identity/v3/users/16c5c900840a4f228792ebd2f32943b0 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:37.658999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:37.660512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:37.675882 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25587bd1-ec62-4f84-878e-2ca771c69458 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:37.762191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:37.773443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:37.774765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:37.775239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:37.775663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] PATH_INFO: `/v3/projects/0fbba9e783384fc8953435e5cb3b07db` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:37.777590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fbba9e783384fc8953435e5cb3b07db)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:37.779598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:37.929653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fbba9e783384fc8953435e5cb3b07db', 'request_id': 'req-95cd6627-d128-4da7-aeb3-0ddc09ed9968'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.938495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fbba9e783384fc8953435e5cb3b07db', 'request_id': 'req-95cd6627-d128-4da7-aeb3-0ddc09ed9968'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:37.948809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fbba9e783384fc8953435e5cb3b07db', 'request_id': 'req-95cd6627-d128-4da7-aeb3-0ddc09ed9968'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:38.035956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] The token cache is being invalidate because project 0fbba9e783384fc8953435e5cb3b07db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:38.036889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:38.043254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-95cd6627-d128-4da7-aeb3-0ddc09ed9968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:38.045309 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1229/2503] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:37 2022] DELETE /identity/v3/projects/0fbba9e783384fc8953435e5cb3b07db => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:38.050375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.050885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.051014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.070899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:38.072203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:38.163808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1275/2504] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:38.175230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.175766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.176280 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.202483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:38.203850 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:38.214922 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1230/2505] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:38.217909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.234753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fddb65-6146-4e76-bd4d-37a11bb7e096 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.239714 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.240495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.240744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.241044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.241824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:38.242139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.243824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.258172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1276/2506] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:38.263647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.273422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ac535-cf90-4321-885f-71ba4d9f4e88 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.278361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.279037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.279361 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.279673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.280384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.282499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.432503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1231/2507] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/projects => generated 399 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:26:38.437492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.449509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e7d3-ae00-4f8b-8e3c-4e1893c2a4c8 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.515491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.516132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.516427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.516696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.517707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.519548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.622814 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1277/2508] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:38.627509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.639769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33259232-8e1c-4e4b-8b2d-df4dcba2be92 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.645323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.646047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.646337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.646609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.647374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.648993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.661002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1232/2509] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:38.667248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.677734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d31a2-be36-4141-9f98-9a1379fa00ef admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.685302 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.685984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.686305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.686586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.687388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.689076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7f65e-5879-42cb-87d7-54872078b721 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.700865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1278/2510] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:38.705333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:38.716403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424fe04-ffe7-4767-94a0-89ad19b68b42 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:38.722793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:38.723812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.724120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.724405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] PATH_INFO: `/v3/projects/c2503e53b95b4c72bbdf0b98ef1c7360/users/8cd77590048b4bd3ba9a79a6b1446e25/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.743350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2503e53b95b4c72bbdf0b98ef1c7360, user_id=8cd77590048b4bd3ba9a79a6b1446e25, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:38.747069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:38.819831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1233/2511] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:38 2022] PUT /identity/v3/projects/c2503e53b95b4c72bbdf0b98ef1c7360/users/8cd77590048b4bd3ba9a79a6b1446e25/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:38.830002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:38.830331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:38.830559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:38.867837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] MFA Rules not processed for user `8cd77590048b4bd3ba9a79a6b1446e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:38.869249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:38.943181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:38.965390 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1279/2512] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:26:38.969624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.060459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7c305-d334-43af-9ea9-a28a3f460706 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.066068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.066762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.067049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.067377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.068087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.070094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.209977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1234/2513] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:38 2022] POST /identity/v3/projects => generated 401 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:39.211366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.224231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d8e6-63c2-4a91-9e8d-9730dca75faf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.335049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.336256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.336742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.337205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.340263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.344339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.467447 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1280/2514] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:39 2022] POST /identity/v3/users => generated 307 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:39.473523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.480920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:39.484714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea974835-9cb6-47c2-b6ef-86f25121576a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.489509 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.490177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.493175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.493175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.493175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.493175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.505518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1235/2515] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:39.509998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.522686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b5d9-80d0-466b-844a-a89684482dbd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.528847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.529743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.530023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.530296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] PATH_INFO: `/v3/projects/84594fe8d27943fe8bd01b8f0ec64421/users/1371a7df2a594c3196ae67f4fef22ea5/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.547731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84594fe8d27943fe8bd01b8f0ec64421, user_id=1371a7df2a594c3196ae67f4fef22ea5, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.550422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.633084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1281/2516] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:39 2022] PUT /identity/v3/projects/84594fe8d27943fe8bd01b8f0ec64421/users/1371a7df2a594c3196ae67f4fef22ea5/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:39.638548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.649734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e493-9347-41eb-9650-eeb450409d6d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.717728 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.718656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.719472 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.719824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.720884 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.723359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a065d-0253-429a-8133-a9f55a62f1c7 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.734964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1236/2517] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:39.739280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.752973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261b9fe-5c7d-4da6-9ed3-ec0d315ac4fc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.757978 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.758993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.759314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.759592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] PATH_INFO: `/v3/projects/84594fe8d27943fe8bd01b8f0ec64421/users/1371a7df2a594c3196ae67f4fef22ea5/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.762582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84594fe8d27943fe8bd01b8f0ec64421, user_id=1371a7df2a594c3196ae67f4fef22ea5, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:39.765619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:39.792981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1282/2518] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:39 2022] PUT /identity/v3/projects/84594fe8d27943fe8bd01b8f0ec64421/users/1371a7df2a594c3196ae67f4fef22ea5/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:39.793179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.793179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.793179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:39.826196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] MFA Rules not processed for user `1371a7df2a594c3196ae67f4fef22ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:39.826196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:39.900812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1237/2519] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:39.912972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:39.917633 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:39.984307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3aa36-edfa-4a89-8a36-a24ca17a4aa9 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:39.988383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:39.989108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:39.989453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:39.989836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:40.001358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:40.002937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f7179-0f11-4d96-b4f7-d7ea99f92f6e service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:40.016339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1283/2520] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:40.324571 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:40.337056 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9040a9-07f4-41fa-b6e1-0ca456ac4893 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:40.346423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:40.347288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:40.347676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:40.347964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:40.447845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:40.447845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2eaa30-06f8-413a-a1df-5e6889ddcdc4 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:40.454046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1238/2521] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:45.532063 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.532813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.532813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] PATH_INFO: `/` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.534236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1284/2522] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:26:45.540065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf22b8-b512-4ed9-8cdd-0c6c7f0afc26 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.540552 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf22b8-b512-4ed9-8cdd-0c6c7f0afc26 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.540637 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf22b8-b512-4ed9-8cdd-0c6c7f0afc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.562868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-b1cf22b8-b512-4ed9-8cdd-0c6c7f0afc26 None None] MFA Rules not processed for user `0a463c3c3f974bae83a6f4b21729f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:45.564148 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b1cf22b8-b512-4ed9-8cdd-0c6c7f0afc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:45.569822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:45.622028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1239/2523] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:45.633724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:45.636432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:45.650263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585472ac-e2a1-48f4-a1f0-6847823f7e91 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:45.655448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:45.656136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.656418 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.656694 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.666221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:45.667748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094141bb-5e2f-48f5-b9fb-d9bad235f099 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:45.676107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1285/2524] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:45.761052 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fb29-a1b5-4b54-ad6e-8448eae4908f None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.761344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fb29-a1b5-4b54-ad6e-8448eae4908f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.761627 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fb29-a1b5-4b54-ad6e-8448eae4908f None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.763147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1240/2525] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:26:45.770084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.771267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.771400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.793790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:45.795107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:45.825519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:45.877066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1286/2526] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:45.889213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777fb29-a1b5-4b54-ad6e-8448eae4908f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:45.901298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777fb29-a1b5-4b54-ad6e-8448eae4908f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:45.905143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:45.905816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:45.906102 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:45.906376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:45.918731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:45.920239 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf8b4-7e50-49f3-bc4b-282bad8af24b service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:45.932984 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1241/2527] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:47.102886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:47.169839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fae80-b4ed-4b86-b503-a7494d8a0ccf None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:47.174796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:47.175904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.176117 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.176354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] PATH_INFO: `/v3/users/8cd77590048b4bd3ba9a79a6b1446e25` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.177656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd77590048b4bd3ba9a79a6b1446e25)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:47.179638 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:47.506316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd77590048b4bd3ba9a79a6b1446e25'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:47.516911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd77590048b4bd3ba9a79a6b1446e25'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:47.530442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd77590048b4bd3ba9a79a6b1446e25'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:47.531353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd77590048b4bd3ba9a79a6b1446e25'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:47.619405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd77590048b4bd3ba9a79a6b1446e25'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:47.629071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1287/2528] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:47 2022] DELETE /identity/v3/users/8cd77590048b4bd3ba9a79a6b1446e25 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:26:47.633660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.634187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.634352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.651828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:47.653385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.718356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1242/2529] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:47.730717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:47.739395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.801862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:47.821269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f553-aefc-4012-a3dd-b975f5a573be admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:47.826008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:47.826873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.827423 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.827859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.850191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:47.850805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:47.863709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1288/2530] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:48.346433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:48.349014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:48.374744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023b476-2672-406b-ab25-ff3ae00d30b6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:48.381111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:48.382059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:48.382347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:48.382628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] PATH_INFO: `/v3/projects/c2503e53b95b4c72bbdf0b98ef1c7360` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:48.384028 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2503e53b95b4c72bbdf0b98ef1c7360)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:48.386021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:48.534124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2503e53b95b4c72bbdf0b98ef1c7360', 'request_id': 'req-e97a748a-a110-4cd2-97bb-afc102796902'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.539722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2503e53b95b4c72bbdf0b98ef1c7360', 'request_id': 'req-e97a748a-a110-4cd2-97bb-afc102796902'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.546357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2503e53b95b4c72bbdf0b98ef1c7360', 'request_id': 'req-e97a748a-a110-4cd2-97bb-afc102796902'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.617243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] The token cache is being invalidate because project c2503e53b95b4c72bbdf0b98ef1c7360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:48.617479 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.618146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.619785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1243/2531] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:48 2022] DELETE /identity/v3/projects/c2503e53b95b4c72bbdf0b98ef1c7360 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:26:48.625450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:48.635669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e35ffd-b4bd-48ba-a0f5-d1718cf06eeb service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:48.703884 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:48.705523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:48.706013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:48.706464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] PATH_INFO: `/v3/users/1371a7df2a594c3196ae67f4fef22ea5` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:48.708129 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1371a7df2a594c3196ae67f4fef22ea5)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:48.711733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:48.971248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1371a7df2a594c3196ae67f4fef22ea5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.979295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1371a7df2a594c3196ae67f4fef22ea5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.996721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1371a7df2a594c3196ae67f4fef22ea5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:48.997769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1371a7df2a594c3196ae67f4fef22ea5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.087988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-2f0ad893-6858-4261-8b8f-42b7406a1519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1371a7df2a594c3196ae67f4fef22ea5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.096379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1289/2532] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:48 2022] DELETE /identity/v3/users/1371a7df2a594c3196ae67f4fef22ea5 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:49.597690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:49.600074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:49.617799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a748a-a110-4cd2-97bb-afc102796902 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:49.657823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:49.689766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:49.690686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:49.690969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:49.691323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] PATH_INFO: `/v3/projects/84594fe8d27943fe8bd01b8f0ec64421` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:49.692525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=84594fe8d27943fe8bd01b8f0ec64421)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:49.694441 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:49.823502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84594fe8d27943fe8bd01b8f0ec64421', 'request_id': 'req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.829610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84594fe8d27943fe8bd01b8f0ec64421', 'request_id': 'req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.835895 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84594fe8d27943fe8bd01b8f0ec64421', 'request_id': 'req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.939043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] The token cache is being invalidate because project 84594fe8d27943fe8bd01b8f0ec64421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:49.939276 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.939926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-dde49519-40c6-4dc4-a9b7-7a621edf0ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:49.941738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1244/2533] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:49 2022] DELETE /identity/v3/projects/84594fe8d27943fe8bd01b8f0ec64421 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:49.946829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:49.947450 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:49.947594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:49.977455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:49.978687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:50.052706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1290/2534] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:26:50.061838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.062036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.062242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.082721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:50.084125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:50.094365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1245/2535] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:50.099258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.105481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:50.112976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef6273-b9fe-4350-a8ff-c041685d3443 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.119975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.119975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.119975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.119975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.120331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:50.120416 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.122863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.139930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1291/2536] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:50.150088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.160445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c434-678a-4539-9f7f-beb1514017da None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.168775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.169501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.169963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.170424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.171234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.173152 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.259924 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1246/2537] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:50 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:50.269913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.281070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e3ea0-82d2-472b-97ea-d976f040d332 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.346395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.347080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.347425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.347797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.348813 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.351982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.466428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1292/2538] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:50 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:50.471615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.488377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d106660-284a-4eee-868e-bc8ece809498 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.493315 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.494043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.494332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.494612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.495414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.498138 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.509561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1247/2539] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:50.514114 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.524973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245e926-64f1-441e-b850-c36085d43458 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.530013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.530684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.530963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.531303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.532095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.533837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af599e5a-5cdd-4b72-9b84-57fb42c4e15e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.546134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1293/2540] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:26:50.550465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.561706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427cc5-854d-437a-aa57-35ad5c5c0998 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.566438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.567432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.571743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.571743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] PATH_INFO: `/v3/projects/6c83a7115708475e9bfe8940f2f8a848/users/a2314d8dce934413908262448249dbf6/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.585176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c83a7115708475e9bfe8940f2f8a848, user_id=a2314d8dce934413908262448249dbf6, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.587920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.693927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1248/2541] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:50 2022] PUT /identity/v3/projects/6c83a7115708475e9bfe8940f2f8a848/users/a2314d8dce934413908262448249dbf6/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:50.705347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.705717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.705964 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.735712 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] MFA Rules not processed for user `a2314d8dce934413908262448249dbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:50.736990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:50.817177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1294/2542] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:50.828994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:50.831145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:50.894777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acd26c-7ab8-4ef4-aedc-3b6d070b9adb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:50.898654 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:50.899368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:50.899656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:50.899938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:50.910458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:50.911962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9268c96-29d8-4ee7-a8fc-59ac83eefd98 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:50.917707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:50.923507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1249/2543] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:50.943091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:51.019835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa3003-ce27-440d-b571-d5852cf6d8ad None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:51.024534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:51.025412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:51.025700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:51.025982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] PATH_INFO: `/v3/users/a2314d8dce934413908262448249dbf6` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:51.028193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2314d8dce934413908262448249dbf6)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:51.029724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:51.031412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:51.231322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2314d8dce934413908262448249dbf6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:51.238819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2314d8dce934413908262448249dbf6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:51.252069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2314d8dce934413908262448249dbf6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:51.252839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2314d8dce934413908262448249dbf6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:51.345068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2314d8dce934413908262448249dbf6'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:51.352072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1295/2544] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:50 2022] DELETE /identity/v3/users/a2314d8dce934413908262448249dbf6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:51.357861 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:51.358395 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:51.358822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:51.380331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:51.381547 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:51.446122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1250/2545] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:51.456230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:51.457702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:51.521188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66df8f-2c9c-4f46-a269-c04d275b857b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:51.525024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:51.525662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:51.525947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:51.526218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:51.538862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:51.538862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053685-4629-4e3a-8d7b-761d9c4af092 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:51.550179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1296/2546] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:26:52.007676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.009290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:52.026380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9184c1-557f-4094-bb1c-a7610756730c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.033495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.034394 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.034709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.034991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] PATH_INFO: `/v3/projects/6c83a7115708475e9bfe8940f2f8a848` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.036577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c83a7115708475e9bfe8940f2f8a848)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.038601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.180418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c83a7115708475e9bfe8940f2f8a848', 'request_id': 'req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:52.187048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c83a7115708475e9bfe8940f2f8a848', 'request_id': 'req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:52.193843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c83a7115708475e9bfe8940f2f8a848', 'request_id': 'req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:52.304600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] The token cache is being invalidate because project 6c83a7115708475e9bfe8940f2f8a848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:52.304857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:52.305500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-1c0e3a03-e2f5-4bea-9d52-faec8282eeec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:52.307114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1251/2547] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:52 2022] DELETE /identity/v3/projects/6c83a7115708475e9bfe8940f2f8a848 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:52.312170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.312677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.312795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.333336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:52.334558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:52.398120 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1297/2548] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:52.405969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.406261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.406576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.425314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:52.426515 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:52.436067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1252/2549] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:52.440357 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.450853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa215f-c368-474c-9669-ec485f216eb8 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.456250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.457001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.457378 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.457733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.458837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:52.458837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.460620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.473846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1298/2550] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:26:52.478035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.488864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09474e4-9ce5-4fcc-ab1c-ea295c4cb08f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.494489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.495188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.495481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.496683 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.497501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.499456 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.595099 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1253/2551] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:52 2022] POST /identity/v3/projects => generated 359 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:52.596457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.610572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b700-535a-4f90-b38c-e9f2bcd81bb0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.619432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:52.671502 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.672219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.672640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.672922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.673753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.676860 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.796267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1299/2552] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:52 2022] POST /identity/v3/users => generated 280 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:52.800796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.818108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eefa9-6d2e-4644-83db-e5b178ab3ad4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.825564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.826932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.827414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.828204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.829543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.831714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.843420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1254/2553] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:52.847412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.858554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5173a3-5be0-42d4-ab5f-8081f6873ec5 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.863232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.863887 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.864169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.864445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.865168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.866759 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d338db-c4af-4a3c-bede-ecb9f275c606 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.880348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1300/2554] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:52.884822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:52.895793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd199-d80e-4e4c-9906-18f2191f68b5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:52.900990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:52.901891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:52.902313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:52.902702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] PATH_INFO: `/v3/projects/9844af7bb58b453c87172748abcb3006/users/f41c9726ad20450cae44dff924257d3d/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:52.919432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9844af7bb58b453c87172748abcb3006, user_id=f41c9726ad20450cae44dff924257d3d, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:52.922147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:52.998188 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1255/2555] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:52 2022] PUT /identity/v3/projects/9844af7bb58b453c87172748abcb3006/users/f41c9726ad20450cae44dff924257d3d/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:53.065421 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:53.065858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:53.066928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:53.102805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] MFA Rules not processed for user `f41c9726ad20450cae44dff924257d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:53.103983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:53.181945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1301/2556] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:53.215593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:53.217461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:53.240419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:53.279730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443cb7d-eee7-4969-ba62-c9299056248f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:53.284069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:53.284734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:53.285020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:53.285298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:53.295208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:53.296696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474acc9-8c37-4434-8deb-32353c90147c service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:53.306105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1256/2557] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:53 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:53.320936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:53.390456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663eed8b-2142-4c3e-a936-fe82e679d57d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:53.395314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:53.396193 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:53.396481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:53.396763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] PATH_INFO: `/v3/users/f41c9726ad20450cae44dff924257d3d` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:53.398012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41c9726ad20450cae44dff924257d3d)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:53.400044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:53.422873 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:53.680523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41c9726ad20450cae44dff924257d3d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:53.689685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41c9726ad20450cae44dff924257d3d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:53.704707 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41c9726ad20450cae44dff924257d3d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:53.705556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41c9726ad20450cae44dff924257d3d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:53.819044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41c9726ad20450cae44dff924257d3d'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:53.827691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1302/2558] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:26:53 2022] DELETE /identity/v3/users/f41c9726ad20450cae44dff924257d3d => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:53.832464 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:53.832756 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:53.833096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:53.851916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:53.853145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:53.922459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1257/2559] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:53.932303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:53.935175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:54.020415 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a203-4504-4d25-864f-7adaa0a7145c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:54.024280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:54.024938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:54.025262 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:54.025500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:54.036234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:54.037682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1699fc9-d487-414c-91bc-561362cd474d service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:54.048481 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1303/2560] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:54.545415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:54.546919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:54.560677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f79b18-53ea-4976-8ac9-90d3ebac4664 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:54.565446 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:54.566356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:54.566645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:54.566927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] PATH_INFO: `/v3/projects/9844af7bb58b453c87172748abcb3006` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:54.568145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9844af7bb58b453c87172748abcb3006)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:54.570191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:54.686042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9844af7bb58b453c87172748abcb3006', 'request_id': 'req-94e039c1-9e46-4749-8461-e638f4a9deea'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:54.693059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9844af7bb58b453c87172748abcb3006', 'request_id': 'req-94e039c1-9e46-4749-8461-e638f4a9deea'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:54.699742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9844af7bb58b453c87172748abcb3006', 'request_id': 'req-94e039c1-9e46-4749-8461-e638f4a9deea'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:54.785772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] The token cache is being invalidate because project 9844af7bb58b453c87172748abcb3006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:26:54.786035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:54.786704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-94e039c1-9e46-4749-8461-e638f4a9deea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:26:54.788270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1258/2561] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:26:54 2022] DELETE /identity/v3/projects/9844af7bb58b453c87172748abcb3006 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:54.797412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:54.798593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:54.799391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:54.822765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:54.825011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:54.935999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1304/2562] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:54.944396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:54.944879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:54.945229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:54.989287 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:54.990019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:55.007146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1259/2563] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:55.013110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.029139 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032db80-8a18-4afb-9d03-5c5a4220220a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.034263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.039177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.057056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1305/2564] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:26:55.061516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.074383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ca5d1-404b-415c-8bdb-aab0ccc1b108 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.081684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.082355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.082635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.082910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.083726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.087929 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.212818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1260/2565] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:26:55 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:55.214168 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.225597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14fca-b9f9-4ec8-a7f9-dbfb296038a0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.255072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:55.299287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.299974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.300259 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.300535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.301339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.303299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.407951 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1306/2566] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:26:55 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:26:55.412434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.423514 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d2fa2-d1fd-48cf-9b2d-eeab9f399ca3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.429000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.429568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.429909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.430192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.430934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.432605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.436816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:55.447063 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1261/2567] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:26:55.451991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.462371 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7905a-3cc1-4644-ac40-4f70bd536c8f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.467410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.472503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.472503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.472503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.472503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.472503 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f4ab4-f277-48da-a60c-a1215e2deea3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.483334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1307/2568] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:26:55.487526 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.499602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd78b9-dc70-43ee-b4f2-59a401bf85b2 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.504532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.505448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.505742 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.506023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] PATH_INFO: `/v3/projects/03f07f376e8444f4ab7ecb88496e5a17/users/176930ba527f45cb93bac7e4496eae31/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.524246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f07f376e8444f4ab7ecb88496e5a17, user_id=176930ba527f45cb93bac7e4496eae31, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.526964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.651970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1262/2569] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:26:55 2022] PUT /identity/v3/projects/03f07f376e8444f4ab7ecb88496e5a17/users/176930ba527f45cb93bac7e4496eae31/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:26:55.662829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.663222 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.663563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.690511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] MFA Rules not processed for user `176930ba527f45cb93bac7e4496eae31`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:26:55.691756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:55.759731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1308/2570] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:26:55.771053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:55.772848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:55.834418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bd90-e039-43be-9e0c-843886bb1ce8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:55.839449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:55.839671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:55.840243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:55.840549 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:55.849773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:55.851306 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:55.861384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1263/2571] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:59.933727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:59.935354 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:00.012317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6851-adb2-4588-9898-34702c256ebb None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:00.016177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:00.016852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:00.017137 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:00.017437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:00.019316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:00.080165 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:00.108380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:00.109833 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:00.122113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1309/2572] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:00.326661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:00.326661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:00.334748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac19c7-ee20-4605-9701-792ed66e59de service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:00.338830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:00.340734 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:00.340971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:00.341226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:00.343095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:00.452087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:00.454647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee30b3-dafc-42f8-bc19-81d099a497ca service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:00.469084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1264/2573] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:01.332331 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:01.345568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06cfae-889f-4c88-818d-0cb8dba2de40 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:01.352227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:01.353727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:01.354289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:01.354799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:01.357861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:01.376425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:01.378774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:01.405030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1310/2574] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:06.666516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366ae9-751a-4fc3-9b44-2499d66b3f0a None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:06.668203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366ae9-751a-4fc3-9b44-2499d66b3f0a None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:06.668471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56366ae9-751a-4fc3-9b44-2499d66b3f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:06.693972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-56366ae9-751a-4fc3-9b44-2499d66b3f0a None None] MFA Rules not processed for user `4ebc96dc6b16411a9adb8e91793cda07`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:06.693972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-56366ae9-751a-4fc3-9b44-2499d66b3f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:06.750222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1265/2575] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:11.390931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=17 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:11.461480 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:11.538285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98a3d5-5409-4fae-a25d-858dc8d4e460 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:11.543704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:11.544587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:11.544866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:11.545140 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] PATH_INFO: `/v3/users/176930ba527f45cb93bac7e4496eae31` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:11.546510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176930ba527f45cb93bac7e4496eae31)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:11.548527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:11.828379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176930ba527f45cb93bac7e4496eae31'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:11.836448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176930ba527f45cb93bac7e4496eae31'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:11.851074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176930ba527f45cb93bac7e4496eae31'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:11.852125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176930ba527f45cb93bac7e4496eae31'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:11.939476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176930ba527f45cb93bac7e4496eae31'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:11.946946 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1311/2576] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:27:11 2022] DELETE /identity/v3/users/176930ba527f45cb93bac7e4496eae31 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:11.951968 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:11.952617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:11.952617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:11.973927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:11.975186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:11.989358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:12.050465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1266/2577] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:12.061362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:12.062859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:12.133150 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d9f3-1988-4fa5-8284-fbcf5bd05fa7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:12.142209 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:12.143339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:12.143812 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:12.144187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:12.158993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:12.160838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4fc75-e1b3-4a1b-ad10-dd52ff1f1738 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:12.180268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1312/2578] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:12.698005 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:12.700690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:12.724951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810be9e-d8b6-4af7-b306-126cde3fb78f None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:12.738263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:12.738397 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:12.738656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:12.738940 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] PATH_INFO: `/v3/projects/03f07f376e8444f4ab7ecb88496e5a17` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:12.740772 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f07f376e8444f4ab7ecb88496e5a17)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:12.742799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:12.873655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f07f376e8444f4ab7ecb88496e5a17', 'request_id': 'req-d876768a-d5b7-44d6-9a46-b573fc13a596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:12.879845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f07f376e8444f4ab7ecb88496e5a17', 'request_id': 'req-d876768a-d5b7-44d6-9a46-b573fc13a596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:12.888393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f07f376e8444f4ab7ecb88496e5a17', 'request_id': 'req-d876768a-d5b7-44d6-9a46-b573fc13a596'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:12.991182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] The token cache is being invalidate because project 03f07f376e8444f4ab7ecb88496e5a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:27:12.992115 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:12.992808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-d876768a-d5b7-44d6-9a46-b573fc13a596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:12.997068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1267/2579] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:27:12 2022] DELETE /identity/v3/projects/03f07f376e8444f4ab7ecb88496e5a17 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:13.002804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.003375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.003802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.027362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:13.028977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:13.050876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:13.112871 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1313/2580] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:13.122041 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.122468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.122852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.148019 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:13.149971 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:13.166677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1268/2581] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:13.172077 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.182744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57282a4-6df6-4cd4-9f95-3c0dcbf44180 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.187635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.188310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.188595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.188899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.189625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:27:13.189938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.191649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.210030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1314/2582] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:27:13.215210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.225762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261559f-8cdc-45ff-9fe8-12ba70a7158c None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.237449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.238214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.238502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.238799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.239540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.241504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.359211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1269/2583] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:27:13 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:27:13.367160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.379612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19144c-b71d-4781-851a-4676789f03ec admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.486647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.488155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.488599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.488962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.490067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.493523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.602260 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1315/2584] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:27:13 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:27:13.608374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.620121 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa69282-f5d0-4b41-b737-dd0c24ba3f6e admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.641297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.643904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.648132 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.649727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.652990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.656323 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.665834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1270/2585] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:27:13.671100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.684979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c55c-3fc0-49f8-b043-1b663c97f3ed admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.689805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.693883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.693883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.693883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.693883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.693883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40075305-4559-40d9-bdeb-6251e3f66ca8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.704522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1316/2586] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:27:13.708794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.719841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332700a-391e-4252-a8d9-300f129ee852 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:13.724541 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:13.725367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.725651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.725931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] PATH_INFO: `/v3/projects/43d24893eba44826a9a4e352bc1a6d88/users/ecb3c92277e2440498b0db94d78dc755/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.742598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d24893eba44826a9a4e352bc1a6d88, user_id=ecb3c92277e2440498b0db94d78dc755, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:13.745342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:13.815197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1271/2587] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:27:13 2022] PUT /identity/v3/projects/43d24893eba44826a9a4e352bc1a6d88/users/ecb3c92277e2440498b0db94d78dc755/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:13.834429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:13.834885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:13.835288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:13.865086 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] MFA Rules not processed for user `ecb3c92277e2440498b0db94d78dc755`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:13.866401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:13.958521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1317/2588] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:13.970531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:13.972449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:14.026500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:14.056562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb201-6bff-42e4-b124-efbf3e65dab9 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:14.062319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:14.062986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:14.063337 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:14.064234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:14.081011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:14.082474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:14.094562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1272/2589] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:23.954321 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:24.127757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=10 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:24.503298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:24.504861 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:24.586550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0400-47e7-4f92-8360-268567a6f07e None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:24.593324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:24.593998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:24.594286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:24.594562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:24.596472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:24.695053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:24.696593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:24.718212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1318/2590] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:27:26.943956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:26.945833 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:27.015025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828261c8-c64c-4406-8010-98c564299c9b service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:27.019162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:27.019919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:27.020162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:27.020440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:27.022316 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:27.033278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:27.034733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152988e-c674-46c1-b037-28af1aebfa58 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:27.044191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1273/2591] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:34.715308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=69 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:37.039665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=26 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:40.630680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:40.712559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6563646-58ca-4ecf-ac54-c451434f4780 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:40.717970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:40.718385 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:40.718671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:40.718948 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] PATH_INFO: `/v3/users/ecb3c92277e2440498b0db94d78dc755` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:40.720899 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb3c92277e2440498b0db94d78dc755)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:40.722158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:40.936198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb3c92277e2440498b0db94d78dc755'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:40.945095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb3c92277e2440498b0db94d78dc755'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:40.959088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb3c92277e2440498b0db94d78dc755'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:40.962159 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb3c92277e2440498b0db94d78dc755'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.057365 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb3c92277e2440498b0db94d78dc755'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.066439 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1319/2592] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:27:40 2022] DELETE /identity/v3/users/ecb3c92277e2440498b0db94d78dc755 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:41.072097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.072739 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.073225 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.094440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:41.095773 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:41.161879 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1274/2593] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:41.171577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:41.173099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:41.239486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335a93-b126-4059-a21e-759da9a66680 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:41.243198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:41.243866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.244158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.244438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.256136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:41.257674 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a53c2-e301-403c-bb82-c1e8b70196e1 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:41.268700 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1320/2594] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:41.736969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:41.738794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:41.754050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81348d8e-c155-438e-848a-e1714830f3d2 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:41.761643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:41.763258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.763657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.763946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] PATH_INFO: `/v3/projects/43d24893eba44826a9a4e352bc1a6d88` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.767043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d24893eba44826a9a4e352bc1a6d88)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:41.769438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:41.883313 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d24893eba44826a9a4e352bc1a6d88', 'request_id': 'req-87ccaa06-d015-4590-a9c5-efb14905c584'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.890788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d24893eba44826a9a4e352bc1a6d88', 'request_id': 'req-87ccaa06-d015-4590-a9c5-efb14905c584'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.898949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d24893eba44826a9a4e352bc1a6d88', 'request_id': 'req-87ccaa06-d015-4590-a9c5-efb14905c584'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.987172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] The token cache is being invalidate because project 43d24893eba44826a9a4e352bc1a6d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:27:41.987643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.988574 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:27:41.990598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1275/2595] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:27:41 2022] DELETE /identity/v3/projects/43d24893eba44826a9a4e352bc1a6d88 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:27:51.263685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:51.981436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:58.382214 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:58.382719 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:58.382823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:58.436014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:58.437457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:58.512205 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1321/2596] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:27:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:58.666954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:58.668780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:58.769716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccaa06-d015-4590-a9c5-efb14905c584 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:58.774818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:58.775503 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:58.775818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:58.776101 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:58.787909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:58.789679 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1bc56-b758-4dbc-9f9c-352e23315604 service swift] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:58.798785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1276/2597] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:27:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:28:08.508542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=40 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:08.794904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=41 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:17.372561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:17.375226 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:17.456033 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642311b4-8b6e-4d38-b17d-292a11588fe9 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:17.461971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:17.462677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:17.463174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:17.463400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:17.465123 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:17.478773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:17.488956 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e72229-dc5e-4e91-a868-6c3f60b60083 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:17.502728 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1322/2598] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:28:19.606484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:19.607055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:19.607055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] PATH_INFO: `/` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:19.608564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1277/2599] 10.222.0.59 () {54 vars in 921 bytes} [Thu Jul 14 11:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:28:19.614132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:19.614590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:19.614711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:19.654983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] MFA Rules not processed for user `c101547390374884b75d95b637267e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:28:19.656428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:19.665905 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1323/2600] 10.222.0.59 () {58 vars in 1018 bytes} [Thu Jul 14 11:28:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:28:19.677659 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:19.679405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:19.759710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5658bf-ed33-4720-bc58-4cc05335d419 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:19.763345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:19.764024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:19.764310 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:19.764711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:19.775598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:19.777114 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f270e42-8c0b-4905-a3f4-84d0034aa41a service glance] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:19.788568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1278/2601] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:20.157039 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:20.158406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:20.234177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0542d-4225-4c06-958a-b831d7564d80 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:20.239109 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:20.239780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:20.240061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:20.240335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:20.242103 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:20.255249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:20.256784 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:20.267731 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1324/2602] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:22.653696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499309-3985-4da3-8eee-f4b58fcd06f9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:22.654165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499309-3985-4da3-8eee-f4b58fcd06f9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:22.654267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb499309-3985-4da3-8eee-f4b58fcd06f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:22.673988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-fb499309-3985-4da3-8eee-f4b58fcd06f9 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:28:22.675259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-fb499309-3985-4da3-8eee-f4b58fcd06f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:22.682631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1279/2603] 10.222.0.59 () {58 vars in 1009 bytes} [Thu Jul 14 11:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:28:22.695149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:22.707091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdbc4d-b3c3-47d9-aeb7-e95dc982110a service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:22.712263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:22.712917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:22.713202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:22.713479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:22.722769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:22.724333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:22.733736 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1325/2604] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:28:32.667201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=38 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:32.729562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=81 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:43.940623 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:43.941112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:43.941213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:43.960253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] MFA Rules not processed for user `54bf77c17b884999a745b47787217a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:28:43.961483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:43.969560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1280/2605] 10.222.0.59 () {58 vars in 967 bytes} [Thu Jul 14 11:28:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:28:43.982386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:43.993698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4797d4-4dd7-42c1-bc66-a4eb48bbab7c service swift] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:43.999042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e631e04eeba421885436f9f2a1330b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:43.999725 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:44.000019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:44.000300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:44.008647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:44.010234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:44.018575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1326/2606] 10.222.0.59 () {60 vars in 1384 bytes} [Thu Jul 14 11:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:28:53.954559 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:54.015328 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:56.688534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:56.692279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:56.709931 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34fae2-253f-4fc1-9ec4-068f1fca0d72 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:56.716670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:56.717548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:56.717926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:56.718270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:56.720141 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:56.733480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:56.735794 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:56.755883 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1281/2607] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:59.958073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:59.970886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26b4dd-e951-46a1-8049-9f9f0cdc2563 service swift] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:59.981823 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:59.982553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:59.982834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:59.983110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:59.985064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:59.999646 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:00.001153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:00.012152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1327/2608] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:02.222177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:02.224258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:02.299204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17778894-1da9-454d-b8c1-d13b905d6693 service nova] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:02.304587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:02.305185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:02.305471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:02.305747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:02.307604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:02.383346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:02.384788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:02.396347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1282/2609] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:10.007456 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:10.708744 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:10.710203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:10.725987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d8ee3-f1c7-4ddc-9df6-7a0ea837db51 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:10.730746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:10.731648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:10.731938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:10.732217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] PATH_INFO: `/v3/users/6b84187a669e4849b3b9d02d8b337824` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:10.733504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b84187a669e4849b3b9d02d8b337824)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:10.735558 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:10.948596 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b84187a669e4849b3b9d02d8b337824'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:10.955821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b84187a669e4849b3b9d02d8b337824'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:10.968975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b84187a669e4849b3b9d02d8b337824'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:10.969757 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b84187a669e4849b3b9d02d8b337824'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.045877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b84187a669e4849b3b9d02d8b337824'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.053726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1328/2610] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:10 2022] DELETE /identity/v3/users/6b84187a669e4849b3b9d02d8b337824 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:11.349505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:11.360847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f096c2-cc3c-4b2f-a15b-8c5995bc3f66 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:11.430885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:11.431802 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:11.432090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:11.432371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] PATH_INFO: `/v3/projects/81b8c45981fe4d48bf95dfbdd0804205` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:11.433701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b8c45981fe4d48bf95dfbdd0804205)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:11.435652 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:11.549243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:11.557062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b8c45981fe4d48bf95dfbdd0804205', 'request_id': 'req-07872630-7a21-471d-9c40-ec269638aefa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.565780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b8c45981fe4d48bf95dfbdd0804205', 'request_id': 'req-07872630-7a21-471d-9c40-ec269638aefa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.572842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b8c45981fe4d48bf95dfbdd0804205', 'request_id': 'req-07872630-7a21-471d-9c40-ec269638aefa'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.670616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] The token cache is being invalidate because project 81b8c45981fe4d48bf95dfbdd0804205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:11.670825 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.671501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:11.673084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1283/2611] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:11 2022] DELETE /identity/v3/projects/81b8c45981fe4d48bf95dfbdd0804205 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:11.677530 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:11.679515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:11.695506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44318be-4cf6-499c-970a-d096fa376885 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:11.774022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:11.774979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:11.775309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:11.775589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] PATH_INFO: `/v3/users/0d9974b0e7984c118c57bee5104f756b` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:11.776828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9974b0e7984c118c57bee5104f756b)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:11.778819 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:11.997916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9974b0e7984c118c57bee5104f756b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.006087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9974b0e7984c118c57bee5104f756b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.018367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9974b0e7984c118c57bee5104f756b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.019745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9974b0e7984c118c57bee5104f756b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.021501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:12.109442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9974b0e7984c118c57bee5104f756b'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.116839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1329/2612] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:11 2022] DELETE /identity/v3/users/0d9974b0e7984c118c57bee5104f756b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:12.379992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:12.380926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:12.394314 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872630-7a21-471d-9c40-ec269638aefa admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:12.462529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:12.464074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:12.464428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:12.464871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] PATH_INFO: `/v3/projects/b23c451f459944fdb64f2a8edd443d07` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:12.466913 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23c451f459944fdb64f2a8edd443d07)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:12.470086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:12.572388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23c451f459944fdb64f2a8edd443d07', 'request_id': 'req-cef7a952-53d6-4785-b967-aa85305b46bd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.578289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23c451f459944fdb64f2a8edd443d07', 'request_id': 'req-cef7a952-53d6-4785-b967-aa85305b46bd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.585642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23c451f459944fdb64f2a8edd443d07', 'request_id': 'req-cef7a952-53d6-4785-b967-aa85305b46bd'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.681596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] The token cache is being invalidate because project b23c451f459944fdb64f2a8edd443d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:12.681886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.682567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:12.684245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1284/2613] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:12 2022] DELETE /identity/v3/projects/b23c451f459944fdb64f2a8edd443d07 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:22.177979 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:22.677786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:41.607797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:41.609377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:41.668073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc21c-ae72-4700-9cd4-6434cdbe5217 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:41.727562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:41.728218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:41.728580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:41.729425 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:41.731329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:41.799198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:41.800548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:41.811564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1330/2614] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:45.618645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:45.621849 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:45.708296 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7a952-53d6-4785-b967-aa85305b46bd admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:45.714387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:45.715051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:45.719791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:45.719791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:45.719791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:45.731676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:45.733372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:45.750992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1285/2615] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:29:45.752230 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:45.755652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:45.770265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba11f8-0746-43fb-8ece-4ccd853c8ca4 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:45.777432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:45.778720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:45.779013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:45.779438 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] PATH_INFO: `/v3/users/4ebc96dc6b16411a9adb8e91793cda07` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:45.795991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebc96dc6b16411a9adb8e91793cda07)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:45.797952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:45.941073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebc96dc6b16411a9adb8e91793cda07'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:45.943909 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:45.950036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebc96dc6b16411a9adb8e91793cda07'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:45.964718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebc96dc6b16411a9adb8e91793cda07'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:45.966267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebc96dc6b16411a9adb8e91793cda07'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.067976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3b3377de-1743-4e3b-9ad9-f1e5cbea0dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebc96dc6b16411a9adb8e91793cda07'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.075373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1331/2616] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:45 2022] DELETE /identity/v3/users/4ebc96dc6b16411a9adb8e91793cda07 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:46.350288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:46.363218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035b1dd-e2d6-4ed4-bb8d-513212547b81 service placement] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:46.435070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:46.436840 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:46.436973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:46.437247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] PATH_INFO: `/v3/projects/6cf0cfc65d914753b1e19db9e238b4dd` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:46.452322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf0cfc65d914753b1e19db9e238b4dd)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:46.454368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:46.629975 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf0cfc65d914753b1e19db9e238b4dd', 'request_id': 'req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.636214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf0cfc65d914753b1e19db9e238b4dd', 'request_id': 'req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.643268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf0cfc65d914753b1e19db9e238b4dd', 'request_id': 'req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.732938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] The token cache is being invalidate because project 6cf0cfc65d914753b1e19db9e238b4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:46.733255 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.748017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-e9548a04-2be2-47e9-b7ed-fa25cb28ba9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:46.750554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1286/2617] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:46 2022] DELETE /identity/v3/projects/6cf0cfc65d914753b1e19db9e238b4dd => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:46.752957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:46.753414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:46.754270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:46.777629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:29:46.779289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:46.856175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1332/2618] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:29:46.864180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:46.864476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:46.864765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:46.891357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:29:46.892649 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:46.911674 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1287/2619] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:29:46.913392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:46.929311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd101606-0e07-490f-a477-c877d768dcf5 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:46.935529 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:46.936221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:46.936660 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:46.936863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:46.937562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:29:46.938064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:46.941787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:46.956721 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1333/2620] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:29:46.960516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:46.970710 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:46.974474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a30796-3cd0-4aa0-b5c2-925d5e59e0c0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:46.983003 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:46.983722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:46.984006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:46.984289 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:46.984988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:46.987183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.078257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1288/2621] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:29:46 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:29:47.080183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.092194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d37eb-7a0f-498d-b3b5-401c3e5613c6 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.159605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.160798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.161089 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.161367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.162196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.164192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.235519 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1334/2622] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:29:47 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:29:47.239860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.252356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d51a-e5e0-48bf-82b5-5edcb1b79db0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.257275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.257946 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.258229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.258504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.259262 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.261766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.272999 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1289/2623] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:29:47.278085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.290311 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac14b-8b1d-4d67-b7ab-ab82a77fb7bc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.298458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.299269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.299642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.300119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.301079 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.302902 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81d96-997a-4347-97dd-d8b266234ff3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.315679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1335/2624] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:29:47.324274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.339830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d2de-96ce-4caa-9e66-ec011d2d9219 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.346070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.347037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.347358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.347636 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] PATH_INFO: `/v3/projects/772b078e91fb499fa732504f49b20b03/users/870fbd4d1d84438da9bf6d72da3b338f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.366889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=772b078e91fb499fa732504f49b20b03, user_id=870fbd4d1d84438da9bf6d72da3b338f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.370027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.425951 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1290/2625] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:29:47 2022] PUT /identity/v3/projects/772b078e91fb499fa732504f49b20b03/users/870fbd4d1d84438da9bf6d72da3b338f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:47.438194 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.438538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.438767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.469548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] MFA Rules not processed for user `870fbd4d1d84438da9bf6d72da3b338f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:29:47.470770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:47.493314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:47.589495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1336/2626] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:29:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:29:47.591602 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.659943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dfe66-5165-4ee4-bed6-b9e270e452fb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.665581 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.666594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.666925 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.667244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.667959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.669885 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:47.764254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1291/2627] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:29:47 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:29:47.769248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:47.783298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c793-d956-4e4e-ae35-7b5d6f051f53 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:47.878886 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:47.879750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:47.880035 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:47.880308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:47.881029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:47.889477 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.025846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1337/2628] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:29:47 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:29:48.026013 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:48.046246 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764500e-dcc4-4840-8b13-1f8f3fe5f291 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:48.053439 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:48.054134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.054425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.054704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.055718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:48.059268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.076767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1292/2629] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:29:48.077976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:48.096710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a79227-7b7e-450d-abbf-9568983296af admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:48.106342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:48.107392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.107770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.108054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] PATH_INFO: `/v3/projects/4232977c10f34300a4ef44aa3c9b3453/users/572548802d5c4bb49d11fa332aaefb1a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.133867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4232977c10f34300a4ef44aa3c9b3453, user_id=572548802d5c4bb49d11fa332aaefb1a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:48.138070 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.221008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1338/2630] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:29:48 2022] PUT /identity/v3/projects/4232977c10f34300a4ef44aa3c9b3453/users/572548802d5c4bb49d11fa332aaefb1a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:48.225129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:48.245568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebb54f-2580-4c39-bf6d-b247a3fbf32f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:48.376809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:48.377714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.378125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.378600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.379542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:48.381476 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6a222-62bb-4551-a167-b8b1cd8c33ed admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.412025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1293/2631] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:29:48.417334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:48.441509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9615-32d2-4bd3-98bc-74112ed52f83 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:48.447913 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:48.448981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.449277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.452040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] PATH_INFO: `/v3/projects/4232977c10f34300a4ef44aa3c9b3453/users/572548802d5c4bb49d11fa332aaefb1a/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.452040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4232977c10f34300a4ef44aa3c9b3453, user_id=572548802d5c4bb49d11fa332aaefb1a, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:48.454361 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.468412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1339/2632] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:29:48 2022] PUT /identity/v3/projects/4232977c10f34300a4ef44aa3c9b3453/users/572548802d5c4bb49d11fa332aaefb1a/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:29:48.479500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418942f0-3957-4ecc-84f0-e3bb1a84b9eb None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.479853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418942f0-3957-4ecc-84f0-e3bb1a84b9eb None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.480134 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418942f0-3957-4ecc-84f0-e3bb1a84b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.525156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-418942f0-3957-4ecc-84f0-e3bb1a84b9eb None None] MFA Rules not processed for user `572548802d5c4bb49d11fa332aaefb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:29:48.525156 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-418942f0-3957-4ecc-84f0-e3bb1a84b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:48.533995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:48.643291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1294/2633] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:29:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:29:48.657550 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:48.659511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:48.764526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2936bf0-870e-4946-95bb-38f94320b54c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:48.770392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:48.772028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:48.772642 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:48.773407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:48.826938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:48.864604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:48.866068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:48.879765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1340/2634] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:50.457410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:50.457410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:50.457410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:50.476042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] MFA Rules not processed for user `c466157931bd41baabb01abbec492b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:29:50.477267 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:50.537048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1295/2635] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:29:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:29:50.968524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:50.970483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:51.035923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718abf-a9cc-4ee9-b612-b78e0f664c5a service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:51.041727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:51.042677 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:51.042962 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:51.043283 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:51.045526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:51.057265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:51.058989 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:51.068590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1341/2636] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:56.364775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:56.366226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:56.380376 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7c79-ef70-4db5-8004-8181b33bebbc None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:56.383749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:56.384686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:56.384686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:56.385075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:56.395910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:56.397374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:56.415222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1296/2637] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:56.657586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:56.666994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3033411-16f1-4a7e-9faf-1382f349ad5d service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:56.673493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:56.676006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:56.676006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:56.676006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:56.677711 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:56.764316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:56.764316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:56.781232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1342/2638] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:00.973662 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:00.975343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:00.990927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d68cc-cb68-43f6-8b7a-119996c015a3 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:00.996382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:00.997305 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:00.999380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:00.999380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] PATH_INFO: `/v3/users/3ac486b2d0ac42f9ba2fed7b17418b7a` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:00.999380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac486b2d0ac42f9ba2fed7b17418b7a)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:01.001442 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:01.208387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac486b2d0ac42f9ba2fed7b17418b7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.209073 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac486b2d0ac42f9ba2fed7b17418b7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.227410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac486b2d0ac42f9ba2fed7b17418b7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.239198 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac486b2d0ac42f9ba2fed7b17418b7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.313285 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-053821d4-39cd-4d8a-9197-304b19e85d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac486b2d0ac42f9ba2fed7b17418b7a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.320489 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1297/2639] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:00 2022] DELETE /identity/v3/users/3ac486b2d0ac42f9ba2fed7b17418b7a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:01.674237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:01.686238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a421-3f13-47cf-a3e1-8e3dd9d234a8 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:01.754977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:01.755921 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:01.756211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:01.756490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] PATH_INFO: `/v3/projects/f33c57605a974b44a62eebbdaba23ade` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:01.757825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33c57605a974b44a62eebbdaba23ade)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:01.759808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:01.769783 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:01.897771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33c57605a974b44a62eebbdaba23ade', 'request_id': 'req-0547da60-b06a-4a96-8caf-955f43e7d46e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:01.903294 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33c57605a974b44a62eebbdaba23ade', 'request_id': 'req-0547da60-b06a-4a96-8caf-955f43e7d46e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:02.016586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33c57605a974b44a62eebbdaba23ade', 'request_id': 'req-0547da60-b06a-4a96-8caf-955f43e7d46e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:02.029349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] The token cache is being invalidate because project f33c57605a974b44a62eebbdaba23ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:02.030255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:02.030824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-0547da60-b06a-4a96-8caf-955f43e7d46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:02.032411 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1343/2640] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:01 2022] DELETE /identity/v3/projects/f33c57605a974b44a62eebbdaba23ade => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:02.037380 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.037888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.038011 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.051424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:02.057904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:02.059241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:02.124086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1298/2641] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:02.132151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.132435 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.132709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.154326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:02.155581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:02.164937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1344/2642] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:02.173178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.184540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb50727-475a-4bd2-813d-4ef1ad65f914 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.189992 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.190744 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.191040 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.191354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.192124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:30:02.192461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.194333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.208774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1299/2643] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:30:02.214855 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.223465 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2841-edfd-49ca-99bf-9a5c41be84f0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.228145 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.228801 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.229202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.230133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.231054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.233160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.349792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1345/2644] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:02.354179 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.367754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97482a95-2adc-411e-a9e3-78dbff4faf7c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.427020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.427020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.427020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.427020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.427601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.429329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.525312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1300/2645] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:02 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:02.531181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.540821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91473b11-2c70-4e73-8323-6f322d9dbf83 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.546308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.558507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.558507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.558507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.558507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.558507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.561349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1346/2646] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:02.567261 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.580990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c572a-eac9-4bfe-b2b9-4eeb42982837 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.586301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.587136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.587457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.587881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.589792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.591304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a22f0-1622-4646-bc10-0329a87636f8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.606872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1301/2647] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:02.608027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.618555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509d00b-2236-4eb0-b856-596de88395cb admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.623821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.632081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.632081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.632081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] PATH_INFO: `/v3/projects/9a3aebe2cb6342ce80d1979e6c330815/users/796d04c1e8a348d9ba3832cb0ede80f3/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.641183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3aebe2cb6342ce80d1979e6c330815, user_id=796d04c1e8a348d9ba3832cb0ede80f3, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.643876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.740931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1347/2648] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:02 2022] PUT /identity/v3/projects/9a3aebe2cb6342ce80d1979e6c330815/users/796d04c1e8a348d9ba3832cb0ede80f3/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:02.748765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2402-fd4a-4fd6-8f89-bca249c3e367 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.748765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2402-fd4a-4fd6-8f89-bca249c3e367 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.748765 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2402-fd4a-4fd6-8f89-bca249c3e367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.778064 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-2b2f2402-fd4a-4fd6-8f89-bca249c3e367 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:02.780301 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-2b2f2402-fd4a-4fd6-8f89-bca249c3e367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:02.862607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1302/2649] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:30:02.881249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:02.883257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:02.952268 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c504-5aeb-4216-b74f-2028b8f7530e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:02.956023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:02.956718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:02.956998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:02.957266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:02.967020 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:02.968490 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:02.981475 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1348/2650] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:08.995093 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63d5fe-565a-4525-8ca9-e6e5b55b373c None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:08.995730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63d5fe-565a-4525-8ca9-e6e5b55b373c None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:08.996712 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63d5fe-565a-4525-8ca9-e6e5b55b373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:09.026998 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:09.030962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-cd63d5fe-565a-4525-8ca9-e6e5b55b373c None None] MFA Rules not processed for user `796d04c1e8a348d9ba3832cb0ede80f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:09.032523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-cd63d5fe-565a-4525-8ca9-e6e5b55b373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:09.140176 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1303/2651] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:09.147889 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:09.149335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:09.210796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:09.250099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc21e01-0ac3-4003-9a6a-42082b0a22da service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:09.255974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d297a-327f-43ef-be83-74b507272878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:09.256645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d297a-327f-43ef-be83-74b507272878 service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:09.256928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d297a-327f-43ef-be83-74b507272878 service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:09.257203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d297a-327f-43ef-be83-74b507272878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:09.267928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d297a-327f-43ef-be83-74b507272878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:09.269515 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d297a-327f-43ef-be83-74b507272878 service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:09.280405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1349/2652] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:12.648721 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:12.649220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:12.649341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:12.676094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] MFA Rules not processed for user `796d04c1e8a348d9ba3832cb0ede80f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:12.677432 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:12.686657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1304/2653] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:13.211319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d297a-327f-43ef-be83-74b507272878 service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:13.216336 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-439d297a-327f-43ef-be83-74b507272878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:13.233803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d297a-327f-43ef-be83-74b507272878 service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:13.237589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:13.238273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:13.239199 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:13.239485 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:13.245679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:13.253912 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:13.255672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:13.266763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1350/2654] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:30:17.757707 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:17.759214 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:17.833365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7596c43-eaef-4982-aba0-37f06895edd1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:17.838642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:17.839967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:17.840370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:17.840511 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:17.842821 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:17.856760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:17.858449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4cf66-499d-4451-af39-25ab219cd391 service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:17.871930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1305/2655] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:23.262526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=24 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:27.865781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=61 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:31.110388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:31.114402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:31.138360 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62f21d-fd9c-428c-acfa-495272ce0617 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:31.144938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:31.145950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:31.146982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:31.147611 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] PATH_INFO: `/v3/users/870fbd4d1d84438da9bf6d72da3b338f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:31.148934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=870fbd4d1d84438da9bf6d72da3b338f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:31.151908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:31.378706 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870fbd4d1d84438da9bf6d72da3b338f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:31.386690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870fbd4d1d84438da9bf6d72da3b338f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:31.401877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870fbd4d1d84438da9bf6d72da3b338f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:31.403780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870fbd4d1d84438da9bf6d72da3b338f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:31.478837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870fbd4d1d84438da9bf6d72da3b338f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:31.485995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1351/2656] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:31 2022] DELETE /identity/v3/users/870fbd4d1d84438da9bf6d72da3b338f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:30:31.492838 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:31.493628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:31.493628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:31.514485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:31.515774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:31.608408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1306/2657] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:31.623305 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:31.624839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:31.694553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305e27-5fa5-4f55-85c4-a78e8efb7c1e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:31.699383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:31.700067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:31.700372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:31.700788 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:31.711849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:31.713482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:31.731278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1352/2658] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:32.280862 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:32.284085 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:32.302775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a768c-b6bc-414e-abcd-2177f77383b6 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:32.310006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:32.310146 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:32.310445 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:32.310731 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] PATH_INFO: `/v3/projects/772b078e91fb499fa732504f49b20b03` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:32.312370 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772b078e91fb499fa732504f49b20b03)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:32.315017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:32.460054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772b078e91fb499fa732504f49b20b03', 'request_id': 'req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:32.467761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772b078e91fb499fa732504f49b20b03', 'request_id': 'req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:32.478110 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772b078e91fb499fa732504f49b20b03', 'request_id': 'req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:32.551433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] The token cache is being invalidate because project 772b078e91fb499fa732504f49b20b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:32.552157 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:32.552585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:32.555595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1307/2659] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:32 2022] DELETE /identity/v3/projects/772b078e91fb499fa732504f49b20b03 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:32.561662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:32.580986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfad51-3d49-4733-907e-9d66907e8b33 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:32.744594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:32.748644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:32.749144 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:32.749618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] PATH_INFO: `/v3/users/572548802d5c4bb49d11fa332aaefb1a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:32.753244 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572548802d5c4bb49d11fa332aaefb1a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:32.769803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:32.800173 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:32.997782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572548802d5c4bb49d11fa332aaefb1a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.007920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572548802d5c4bb49d11fa332aaefb1a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.022376 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572548802d5c4bb49d11fa332aaefb1a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.027085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572548802d5c4bb49d11fa332aaefb1a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.102201 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33f51692-b85d-46b1-a3a9-205740391164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572548802d5c4bb49d11fa332aaefb1a'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.114108 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1353/2660] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:32 2022] DELETE /identity/v3/users/572548802d5c4bb49d11fa332aaefb1a => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:33.606586 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:33.611569 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:33.625997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b7a16-1fe0-401e-a9ad-8193bf9f2d25 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:33.707866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:33.708812 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:33.709108 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:33.709393 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] PATH_INFO: `/v3/projects/4232977c10f34300a4ef44aa3c9b3453` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:33.710651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4232977c10f34300a4ef44aa3c9b3453)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:33.712754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:33.817530 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:33.837106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4232977c10f34300a4ef44aa3c9b3453', 'request_id': 'req-9e7a7904-5938-4ac5-b699-54ff26a6cca5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.844292 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4232977c10f34300a4ef44aa3c9b3453', 'request_id': 'req-9e7a7904-5938-4ac5-b699-54ff26a6cca5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.857504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4232977c10f34300a4ef44aa3c9b3453', 'request_id': 'req-9e7a7904-5938-4ac5-b699-54ff26a6cca5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.935281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] The token cache is being invalidate because project 4232977c10f34300a4ef44aa3c9b3453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:33.935611 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.936607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-9e7a7904-5938-4ac5-b699-54ff26a6cca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:33.941846 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1308/2661] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:33 2022] DELETE /identity/v3/projects/4232977c10f34300a4ef44aa3c9b3453 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:33.943680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:33.944172 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:33.944261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:33.970927 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:33.973523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:34.059601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1354/2662] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:34.068424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.068715 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.071520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.095328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:34.096601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:34.117678 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.118624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1309/2663] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:34.130263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe234ad-c14c-4d46-a26e-bce4d9f1ad3b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.136710 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.138676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.138676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.138676 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.139293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:30:34.139692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.141772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.158043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1355/2664] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:30:34.162754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.176544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e547d41-1127-4110-9756-8ce945ecdfa4 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.183625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.184523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.185863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.185863 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.186444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.195510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.362626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1310/2665] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:34 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:34.366793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.377998 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1420283-9b36-4a8d-8726-6d558a555b60 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.443973 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.444655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.444940 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.445219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.445924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.448004 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.571457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1356/2666] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:34 2022] POST /identity/v3/users => generated 284 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:34.576568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.589219 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218b17-5456-4ec7-bcce-c8fc3905cf03 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.598183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.600631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.600631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.600631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.600631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.602433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.618799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1311/2667] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:30:34.625999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.639395 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad93090-ee83-4a18-a687-1e4e5a0c04f7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.644662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.645358 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.645643 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.645922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.646661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.648348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f5f65-ca60-42ad-b10e-11cca070c830 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.654859 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:34.662521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1357/2668] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:34.666933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:34.678750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a51bf-fb79-44f8-b240-be29d6da152d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:34.685920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:34.686905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.687236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.687572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] PATH_INFO: `/v3/projects/da64684fe9794d3e9b8edc88d1efacd0/users/007e49c5b5df47348e095107504755a2/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.707335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da64684fe9794d3e9b8edc88d1efacd0, user_id=007e49c5b5df47348e095107504755a2, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:34.710027 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:34.825740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1312/2669] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:34 2022] PUT /identity/v3/projects/da64684fe9794d3e9b8edc88d1efacd0/users/007e49c5b5df47348e095107504755a2/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:34.836924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:34.837344 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:34.837779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:34.873618 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] MFA Rules not processed for user `007e49c5b5df47348e095107504755a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:34.874972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:34.948816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1358/2670] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:34.953385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.036167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea466e-cdff-4ddd-8f5f-6e75bbca0220 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.041407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.042880 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.043204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.043582 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.044196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.046122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.195084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1313/2671] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:34 2022] POST /identity/v3/projects => generated 367 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:35.199691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.227745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3aa93-33be-4f72-a76f-357fa6cd2475 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.299348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.300028 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.300315 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.300599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.301301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.303446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.382116 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1359/2672] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:35 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:30:35.386769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.401793 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035b4e6-b1c8-4844-bbc3-c807a4502bab admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.412055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.412758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.413074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.413357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.414130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.415788 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.432666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1314/2673] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:35.436954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.457090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13892a03-3267-443d-87ae-e0732bb7dd42 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.461993 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.462891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.463842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.464107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] PATH_INFO: `/v3/projects/3a456e90bb4f42999078fe8acacd484e/users/8b24b8d657c24ae296e14523b14d8973/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.482639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a456e90bb4f42999078fe8acacd484e, user_id=8b24b8d657c24ae296e14523b14d8973, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.487915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.571287 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1360/2674] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:35 2022] PUT /identity/v3/projects/3a456e90bb4f42999078fe8acacd484e/users/8b24b8d657c24ae296e14523b14d8973/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:35.577631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.594546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382facf5-b6b3-4ef9-98ec-26ea10837659 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.618873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:35.668443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.669140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.669433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.669716 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.670462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.672207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a8ea-303f-490c-a788-0cd18496616a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.683717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1315/2675] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:35.687748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.702015 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f75be-3a57-4e50-8024-a88da8a8ef4a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.705852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.706750 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.707037 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.707453 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] PATH_INFO: `/v3/projects/3a456e90bb4f42999078fe8acacd484e/users/8b24b8d657c24ae296e14523b14d8973/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.710265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a456e90bb4f42999078fe8acacd484e, user_id=8b24b8d657c24ae296e14523b14d8973, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.712984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:35.735997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1361/2676] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:35 2022] PUT /identity/v3/projects/3a456e90bb4f42999078fe8acacd484e/users/8b24b8d657c24ae296e14523b14d8973/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:30:35.749164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.749671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.750029 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.786488 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] MFA Rules not processed for user `8b24b8d657c24ae296e14523b14d8973`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:35.787962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:35.878382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1316/2677] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:35.893029 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:35.895460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:35.957235 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:35.972073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae13ad-2345-4ec9-be66-990f67fdd492 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:35.975904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:35.976559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:35.976850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:35.977130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:35.989115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:35.990572 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:36.005392 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1362/2678] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:36.304656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:36.320420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86bd8a-d5da-4b08-a307-193e07c98fe3 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:36.326076 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:36.326753 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:36.327034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:36.327364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:36.425767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:36.427284 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778341c-6682-45c9-b537-580abe265650 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:36.448119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1317/2679] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:30:44.242867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:44.309091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5071-f2d3-4ec3-8ef5-86f20deaf067 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:44.313659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:44.314525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:44.314803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:44.315078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] PATH_INFO: `/v3/users/007e49c5b5df47348e095107504755a2` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:44.316280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=007e49c5b5df47348e095107504755a2)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:44.318180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:44.475316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '007e49c5b5df47348e095107504755a2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:44.482511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '007e49c5b5df47348e095107504755a2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:44.497539 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '007e49c5b5df47348e095107504755a2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:44.498233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '007e49c5b5df47348e095107504755a2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:44.563925 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '007e49c5b5df47348e095107504755a2'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:44.570482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1363/2680] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:44 2022] DELETE /identity/v3/users/007e49c5b5df47348e095107504755a2 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:30:44.575733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:44.580540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:44.580540 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:44.594401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:44.595658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:44.637528 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=100 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:44.657036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1318/2681] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:44.668375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:44.669834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:44.730649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4e59-4056-40c2-b2f9-aecad7830bda admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:44.734499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:44.735202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:44.735507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:44.735816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:44.746511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:44.748066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:44.758203 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1364/2682] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:30:45.184961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:45.187356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:45.203562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee61f6-8b9c-48aa-96a3-a780b956f730 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:45.211544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:45.213465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:45.213956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:45.214347 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] PATH_INFO: `/v3/projects/da64684fe9794d3e9b8edc88d1efacd0` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:45.215527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da64684fe9794d3e9b8edc88d1efacd0)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:45.217632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:45.424782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da64684fe9794d3e9b8edc88d1efacd0', 'request_id': 'req-0a25630c-fee7-4a3d-a30a-d53635b67ed1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.430463 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da64684fe9794d3e9b8edc88d1efacd0', 'request_id': 'req-0a25630c-fee7-4a3d-a30a-d53635b67ed1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.437037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da64684fe9794d3e9b8edc88d1efacd0', 'request_id': 'req-0a25630c-fee7-4a3d-a30a-d53635b67ed1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.506778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] The token cache is being invalidate because project da64684fe9794d3e9b8edc88d1efacd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:45.507158 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.507751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.509342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1319/2683] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:45 2022] DELETE /identity/v3/projects/da64684fe9794d3e9b8edc88d1efacd0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:45.515111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:45.527733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f167935-6cfa-4d70-8483-9820920030b0 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:45.531507 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:45.589635 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:45.590524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:45.590853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:45.591373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] PATH_INFO: `/v3/users/8b24b8d657c24ae296e14523b14d8973` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:45.593182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b24b8d657c24ae296e14523b14d8973)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:45.595274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:45.790106 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b24b8d657c24ae296e14523b14d8973'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.798026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b24b8d657c24ae296e14523b14d8973'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.813009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b24b8d657c24ae296e14523b14d8973'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.813776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b24b8d657c24ae296e14523b14d8973'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.883723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-3cd6413b-61c6-413b-8738-f7f47ef66388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b24b8d657c24ae296e14523b14d8973'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:45.891514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1365/2684] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:45 2022] DELETE /identity/v3/users/8b24b8d657c24ae296e14523b14d8973 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:46.380786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:46.385321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:46.398332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25630c-fee7-4a3d-a30a-d53635b67ed1 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:46.462356 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:46.464053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:46.464454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:46.464737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] PATH_INFO: `/v3/projects/3a456e90bb4f42999078fe8acacd484e` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:46.465949 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a456e90bb4f42999078fe8acacd484e)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:46.467901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:46.571749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a456e90bb4f42999078fe8acacd484e', 'request_id': 'req-daadc735-a233-410f-864f-6dc91e21616f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:46.580495 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a456e90bb4f42999078fe8acacd484e', 'request_id': 'req-daadc735-a233-410f-864f-6dc91e21616f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:46.591069 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a456e90bb4f42999078fe8acacd484e', 'request_id': 'req-daadc735-a233-410f-864f-6dc91e21616f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:46.668259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:46.674916 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] The token cache is being invalidate because project 3a456e90bb4f42999078fe8acacd484e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:46.680996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:46.680996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-daadc735-a233-410f-864f-6dc91e21616f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:46.680996 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1320/2685] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:46 2022] DELETE /identity/v3/projects/3a456e90bb4f42999078fe8acacd484e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:46.685491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:46.686019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:46.686122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:46.705916 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:46.707141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:46.772436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1366/2686] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:46.791317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:46.791644 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:46.791964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:46.811071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:46.812333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:46.820477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1321/2687] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:46.825817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:46.835906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338c0f4-148c-464d-836c-a2cfc316aa70 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:46.841949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:46.842614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:46.842897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:46.843204 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:46.843928 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:30:46.844236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:46.845836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:46.859592 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1367/2688] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:30:46.867485 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:46.878017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d64f70-4e80-435b-9ee1-2d2f141daad8 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:46.882758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:46.883459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:46.883792 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:46.884074 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:46.885295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:46.887244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:46.974145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1322/2689] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:46 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:46.976644 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:46.987936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dde362-312a-4efe-8283-a623621fa682 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:46.997981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:47.051781 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.052458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.052746 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.053025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.053772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.055778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.130267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1368/2690] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:46 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:30:47.135304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.149711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cc583-b7e9-4e40-8b15-4fa88bab27c5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.155097 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.155810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.156096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.156377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.157120 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.158766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.170125 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1323/2691] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:30:47.174603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.187684 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20462d-06f5-482d-9d7b-63486913c0b7 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.191617 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.192654 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.192952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.193238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.194540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.197923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d582f-368c-4148-a730-def07c23aaa3 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.209922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1369/2692] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:30:47.217545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.230700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab2d1-a8de-4e10-a2b7-5d1d9d87e967 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.234130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.235033 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.235351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.235631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] PATH_INFO: `/v3/projects/6dc860a2624442ee8329345fa0d09908/users/00b9d2e2bf434fd8b82709c1942d06b7/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.253058 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc860a2624442ee8329345fa0d09908, user_id=00b9d2e2bf434fd8b82709c1942d06b7, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.258451 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.406377 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1324/2693] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:47 2022] PUT /identity/v3/projects/6dc860a2624442ee8329345fa0d09908/users/00b9d2e2bf434fd8b82709c1942d06b7/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:30:47.417624 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.417922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.418211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.446527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] MFA Rules not processed for user `00b9d2e2bf434fd8b82709c1942d06b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:47.448449 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:47.518355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1370/2694] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:30:47.524527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.590617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11e30c-e613-4f25-b056-5e464c731293 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.595501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.596166 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.596449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.596722 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.597404 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.599342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.693065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1325/2695] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:47 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:47.697942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.709659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd68e7-c943-4965-a7f5-6d23cf1e1dda None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.782455 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.783591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.784046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.784406 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.785398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.788688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.903261 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1371/2696] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:47 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:47.907208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.918191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b27c4-41f0-4dcc-a2b7-9a0485ebaad4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.923698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.924727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.925023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.925486 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.926601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.929164 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:47.942597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1326/2697] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:47.951716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:47.963786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742300-5a5e-47bb-9c23-816813c8f56e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:47.969286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:47.970335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:47.970761 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:47.971160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] PATH_INFO: `/v3/projects/228d9fcdb2f84143be444b2b376724e5/users/3d21d6edfc6f41d5895ffdaeeaa9a28c/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:47.981271 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:47.988111 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228d9fcdb2f84143be444b2b376724e5, user_id=3d21d6edfc6f41d5895ffdaeeaa9a28c, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:47.992511 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:48.067877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1372/2698] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:47 2022] PUT /identity/v3/projects/228d9fcdb2f84143be444b2b376724e5/users/3d21d6edfc6f41d5895ffdaeeaa9a28c/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:48.071923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:48.084434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98222267-be83-4342-84a6-aa6d4ad565d0 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:48.151223 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:48.152159 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:48.152693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:48.153191 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:48.154288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:48.156673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5a53-5e48-49f9-923d-c328ae601061 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:48.173317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1327/2699] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:48.177891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:48.194170 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb35d7-533e-4e06-a846-dd550d3e3180 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:48.201099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:48.202126 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:48.202523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:48.202903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] PATH_INFO: `/v3/projects/228d9fcdb2f84143be444b2b376724e5/users/3d21d6edfc6f41d5895ffdaeeaa9a28c/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:48.205085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228d9fcdb2f84143be444b2b376724e5, user_id=3d21d6edfc6f41d5895ffdaeeaa9a28c, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:48.207745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:48.218671 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1373/2700] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:48 2022] PUT /identity/v3/projects/228d9fcdb2f84143be444b2b376724e5/users/3d21d6edfc6f41d5895ffdaeeaa9a28c/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:48.238167 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:48.238563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:48.238856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:48.247334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:48.268343 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] MFA Rules not processed for user `3d21d6edfc6f41d5895ffdaeeaa9a28c`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:48.269566 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:48.343762 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1328/2701] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:48.356002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:48.357735 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:48.422693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098290f3-da72-4d91-88c1-a23f8a890833 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:48.426545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:48.427252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:48.427659 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:48.427869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:48.438213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:48.439720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb504fd-acae-4e5f-b966-fa2c39fe4c24 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:48.449922 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1374/2702] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:49.624331 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:49.691263 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22170163-ce70-457a-bd7f-0743fd863c77 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:49.696075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:49.697094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:49.697382 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:49.698185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] PATH_INFO: `/v3/users/00b9d2e2bf434fd8b82709c1942d06b7` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:49.699506 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b9d2e2bf434fd8b82709c1942d06b7)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:49.701449 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:49.854699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b9d2e2bf434fd8b82709c1942d06b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:49.855340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b9d2e2bf434fd8b82709c1942d06b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:49.868945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b9d2e2bf434fd8b82709c1942d06b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:49.876958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b9d2e2bf434fd8b82709c1942d06b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:49.940473 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b9d2e2bf434fd8b82709c1942d06b7'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:49.947381 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1329/2703] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:49 2022] DELETE /identity/v3/users/00b9d2e2bf434fd8b82709c1942d06b7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:49.952301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:49.952840 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:49.952970 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:49.970765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:49.972013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:50.038016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1375/2704] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:50.048421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:50.049839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:50.062427 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:50.113558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5388dd-2911-49ca-ad97-c2ef6d1b3f7f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:50.117264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:50.118162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:50.118444 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:50.118718 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:50.129962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:50.131587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:50.141851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1330/2705] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:50.561053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:50.562516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:50.575380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f184b5-d40b-4fb9-a095-fdc8a43fe75f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:50.580155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:50.581023 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:50.581303 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:50.581575 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] PATH_INFO: `/v3/projects/6dc860a2624442ee8329345fa0d09908` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:50.582796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc860a2624442ee8329345fa0d09908)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:50.584764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:50.707821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc860a2624442ee8329345fa0d09908', 'request_id': 'req-25cc1814-5328-4201-8f9c-acaa83b4553c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:50.714524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc860a2624442ee8329345fa0d09908', 'request_id': 'req-25cc1814-5328-4201-8f9c-acaa83b4553c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:50.781682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc860a2624442ee8329345fa0d09908', 'request_id': 'req-25cc1814-5328-4201-8f9c-acaa83b4553c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:50.790627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:50.795026 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] The token cache is being invalidate because project 6dc860a2624442ee8329345fa0d09908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:50.795348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:50.796013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:50.797600 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1376/2706] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:50 2022] DELETE /identity/v3/projects/6dc860a2624442ee8329345fa0d09908 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:50.801610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:50.813522 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b73f46-e5f9-4c18-aa35-90a2c710318e service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:50.874955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:50.875898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:50.876182 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:50.876458 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] PATH_INFO: `/v3/users/3d21d6edfc6f41d5895ffdaeeaa9a28c` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:50.877748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d21d6edfc6f41d5895ffdaeeaa9a28c)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:50.879766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:51.085912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d21d6edfc6f41d5895ffdaeeaa9a28c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.086607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d21d6edfc6f41d5895ffdaeeaa9a28c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.100702 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d21d6edfc6f41d5895ffdaeeaa9a28c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.108847 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d21d6edfc6f41d5895ffdaeeaa9a28c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.163655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-28325d45-f8cd-4c9e-8f38-717e9e21fa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d21d6edfc6f41d5895ffdaeeaa9a28c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.171709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1331/2707] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:50 2022] DELETE /identity/v3/users/3d21d6edfc6f41d5895ffdaeeaa9a28c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:51.606242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:51.612067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:51.624084 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc1814-5328-4201-8f9c-acaa83b4553c admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:51.684521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:51.685379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:51.685665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:51.685944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] PATH_INFO: `/v3/projects/228d9fcdb2f84143be444b2b376724e5` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:51.687146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228d9fcdb2f84143be444b2b376724e5)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:51.689083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:51.805409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228d9fcdb2f84143be444b2b376724e5', 'request_id': 'req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.810934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228d9fcdb2f84143be444b2b376724e5', 'request_id': 'req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.872177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228d9fcdb2f84143be444b2b376724e5', 'request_id': 'req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.883890 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] The token cache is being invalidate because project 228d9fcdb2f84143be444b2b376724e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:51.886842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.886842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-33e0f9c2-3b8d-4e24-bb47-696caa324ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:51.886842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1377/2708] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:51 2022] DELETE /identity/v3/projects/228d9fcdb2f84143be444b2b376724e5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:51.891603 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:51.892090 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:51.892319 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:51.913207 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:51.914811 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:51.942455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:51.977867 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1332/2709] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:51.986697 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:51.986982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:51.987296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.005842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:52.007183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:52.016986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1378/2710] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:30:52.021332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.033329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b61c95-8a1d-45f8-be56-fb99e0aee045 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.038590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.039297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.039592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.039874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.040678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:30:52.041001 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.042613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.057067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1333/2711] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:30:52.061404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.070799 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11ad8-89b9-4893-945a-9d7af9709104 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.076690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.077338 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.077620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.077895 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.078602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.080526 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.164846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1379/2712] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:52 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:52.169046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.179782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f350048-ecd2-4653-ae7b-ed8a68a9f517 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.239412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.240105 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.240368 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.240684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.241438 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.243752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.342500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1334/2713] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:52 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:52.346932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.357804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bd26b-a756-4ce0-9d2d-8f4e982ec007 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.362877 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.363593 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.363883 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.364166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.364903 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.366501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.390457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1380/2714] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:52.394564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.405554 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219efc5d-66cd-4a27-9c96-e8864be9ff9b admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.410259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.410923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.411241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.411519 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.412243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.413842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d84197-d28d-4868-abf4-4f8d0183b100 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.425083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1335/2715] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:52.435066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.445854 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac811ae-f441-40bf-b60f-e4edb8ef740a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.451281 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.452161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.452445 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.452720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] PATH_INFO: `/v3/projects/82fd9e53a9264f1083dd985924169f7b/users/8bedb4165c49459f924cb341fc22b100/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.468312 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82fd9e53a9264f1083dd985924169f7b, user_id=8bedb4165c49459f924cb341fc22b100, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.470954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.569764 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1381/2716] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:52 2022] PUT /identity/v3/projects/82fd9e53a9264f1083dd985924169f7b/users/8bedb4165c49459f924cb341fc22b100/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:52.583095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.583539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.583932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.612422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] MFA Rules not processed for user `8bedb4165c49459f924cb341fc22b100`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:52.613888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:52.682044 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1336/2717] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:52.693648 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.695636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:52.737796 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:52.769233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb076c0-406f-47a6-9781-ad2f709e36dc admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.773152 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.773795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.774078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.774356 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.783053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.784525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aee596-b3b5-44ad-94b9-5f19e1e7c1ac service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:52.795897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1382/2718] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:52.815666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:52.846336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:52.892750 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62d5c1-9af5-4b16-8969-cf1552768b92 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:52.897749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:52.898634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:52.898920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:52.899232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] PATH_INFO: `/v3/users/8bedb4165c49459f924cb341fc22b100` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:52.901118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bedb4165c49459f924cb341fc22b100)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:52.902709 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:53.080317 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bedb4165c49459f924cb341fc22b100'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.080980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bedb4165c49459f924cb341fc22b100'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.097538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bedb4165c49459f924cb341fc22b100'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.107727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bedb4165c49459f924cb341fc22b100'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.175230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bedb4165c49459f924cb341fc22b100'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.185192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1337/2719] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:30:52 2022] DELETE /identity/v3/users/8bedb4165c49459f924cb341fc22b100 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:53.191932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:53.192402 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:53.192881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:53.214664 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:53.215914 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:53.279932 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1383/2720] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:53.290339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:53.292995 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:53.354808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280380c0-eeb5-4ca1-89b3-f7061c7b7209 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:53.358525 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:53.359230 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:53.359512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:53.359784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:53.369873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:53.371401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0306a-04c9-4669-abb9-a9e3f47ef34e service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:53.381901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1338/2721] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:30:53.840541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:53.842105 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:53.857825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34efd6-bd4f-49df-9061-a89898397076 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:53.862446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:53.863381 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:53.863669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:53.863949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] PATH_INFO: `/v3/projects/82fd9e53a9264f1083dd985924169f7b` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:53.865169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82fd9e53a9264f1083dd985924169f7b)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:53.867072 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:53.964837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82fd9e53a9264f1083dd985924169f7b', 'request_id': 'req-991b81a9-928c-441d-b8b4-82b6a16181b9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:53.970410 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82fd9e53a9264f1083dd985924169f7b', 'request_id': 'req-991b81a9-928c-441d-b8b4-82b6a16181b9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:54.027080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82fd9e53a9264f1083dd985924169f7b', 'request_id': 'req-991b81a9-928c-441d-b8b4-82b6a16181b9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:54.039064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] The token cache is being invalidate because project 82fd9e53a9264f1083dd985924169f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:30:54.039427 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:54.040196 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-991b81a9-928c-441d-b8b4-82b6a16181b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:30:54.042794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1384/2722] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:30:53 2022] DELETE /identity/v3/projects/82fd9e53a9264f1083dd985924169f7b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:54.048123 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.048616 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.048730 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.068433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:54.069666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:54.114828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:54.131088 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1339/2723] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:54.195858 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.196149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.196430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.218472 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:54.219726 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:54.228181 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1385/2724] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:54.232619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.243042 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827fa8a-1c20-4a21-a795-f86318f244ed None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.258196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.258196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.258196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.258196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.259693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:30:54.259693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.259693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.265755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1340/2725] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:30:54.270386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.280257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48db3ca-0cde-420a-a2f9-6c72578f2964 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.287908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.288661 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.288944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.289224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.289923 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.292430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.388369 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1386/2726] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:54.392755 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.403944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f3e46-9312-4c7b-a533-e6e34188dfb5 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.467304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.467997 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.468283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.468562 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.469272 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.471244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.586844 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1341/2727] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:30:54 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:30:54.591790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.602233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219878e-994b-459e-8160-325d04b2018e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.608975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.610818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.611601 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.612191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.614870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.620011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.632208 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1387/2728] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:54.636440 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.647007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29817878-6cfe-4aa9-bc9e-2396d5ac944a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.652717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.653392 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.653677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.653954 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.654682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.656338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a9f00-d195-4003-b11c-5ad5219dd9ed admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.667698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1342/2729] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:30:54.672132 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.682931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bacbed-3cd9-4ab0-88df-4bd5d4d532ee admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.688662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.689583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.689863 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.690138 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] PATH_INFO: `/v3/projects/3d7f5688aac141fda1177c689d5bcb7a/users/bad633fb995f48eebfc558449ccdf188/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.705692 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:54.709947 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7f5688aac141fda1177c689d5bcb7a, user_id=bad633fb995f48eebfc558449ccdf188, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:54.713187 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:54.803785 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1388/2730] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:30:54 2022] PUT /identity/v3/projects/3d7f5688aac141fda1177c689d5bcb7a/users/bad633fb995f48eebfc558449ccdf188/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:54.814612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.814907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.815242 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.843000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] MFA Rules not processed for user `bad633fb995f48eebfc558449ccdf188`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:30:54.844286 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:54.910668 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1343/2731] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:30:54.921348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:30:54.923270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:30:54.984641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2653c9-4929-4b91-bd61-e5947c9b1832 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:30:54.988374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:30:54.989014 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:30:54.989313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:30:54.989608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:30:54.999011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:30:55.000549 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d23356-5529-4bb4-a5ed-0897e0c6bd27 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:30:55.011486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1389/2732] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:30:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:31:04.907619 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:05.005537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:06.092422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:06.157617 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08ed2-73a8-4991-bef1-b846c1819a86 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:06.162128 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:06.163327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:06.163609 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:06.163973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] PATH_INFO: `/v3/users/bad633fb995f48eebfc558449ccdf188` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:06.165189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad633fb995f48eebfc558449ccdf188)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:06.167465 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:06.416896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad633fb995f48eebfc558449ccdf188'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:06.417990 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad633fb995f48eebfc558449ccdf188'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:06.430939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad633fb995f48eebfc558449ccdf188'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:06.438853 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad633fb995f48eebfc558449ccdf188'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:06.517249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad633fb995f48eebfc558449ccdf188'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:06.523904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1344/2733] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:31:06 2022] DELETE /identity/v3/users/bad633fb995f48eebfc558449ccdf188 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:31:06.528758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:06.532257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:06.532257 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:06.547875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:31:06.549075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:06.609532 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1390/2734] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:31:06.619795 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:06.621405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:06.684086 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d3e4-13fb-4866-b059-12e92526b8f3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:06.687647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:06.688295 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:06.688699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:06.688985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:06.698247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:06.700567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d4aaf-9740-4ca6-bd98-1284c6b04089 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:06.709944 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1345/2735] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:31:07.142278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.143847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:07.156243 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eb9b3-f6d9-4771-b4b9-1123ffc60774 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.161238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.162110 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.162399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.162679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] PATH_INFO: `/v3/projects/3d7f5688aac141fda1177c689d5bcb7a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.163990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7f5688aac141fda1177c689d5bcb7a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.166027 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.278272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7f5688aac141fda1177c689d5bcb7a', 'request_id': 'req-4f860c7c-a81e-4357-829f-a8c063880fdb'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:07.284161 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7f5688aac141fda1177c689d5bcb7a', 'request_id': 'req-4f860c7c-a81e-4357-829f-a8c063880fdb'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:07.374112 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7f5688aac141fda1177c689d5bcb7a', 'request_id': 'req-4f860c7c-a81e-4357-829f-a8c063880fdb'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:07.386298 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] The token cache is being invalidate because project 3d7f5688aac141fda1177c689d5bcb7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:31:07.386589 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:07.387428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-4f860c7c-a81e-4357-829f-a8c063880fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:07.389010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1391/2736] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:31:07 2022] DELETE /identity/v3/projects/3d7f5688aac141fda1177c689d5bcb7a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:31:07.394196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13708684-a927-4856-924b-823ffbf82a84 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.394675 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13708684-a927-4856-924b-823ffbf82a84 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.394776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13708684-a927-4856-924b-823ffbf82a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.416352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-13708684-a927-4856-924b-823ffbf82a84 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:31:07.417915 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-13708684-a927-4856-924b-823ffbf82a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:07.440140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:07.486189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1346/2737] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:31:07.494219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.494514 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.494821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.514542 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:31:07.515787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:07.524353 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1392/2738] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:31:07.531065 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13708684-a927-4856-924b-823ffbf82a84 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.541243 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13708684-a927-4856-924b-823ffbf82a84 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.547401 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.548087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.548371 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.548647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.549367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:31:07.549678 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.551588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.563859 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1347/2739] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:31:07.568242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.577772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb340c-2b01-4cb5-beba-8d999a27e77b None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.582430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.587285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.587285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.587285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.587285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.587285 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.662276 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1393/2740] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:31:07 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:31:07.666624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.677543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529107-dd61-4dbb-b2a8-514ca292e459 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.740354 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.741062 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.741588 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.741870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.742809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.744910 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.820974 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1348/2741] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:31:07 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:31:07.826091 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.838245 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b2459-1a8f-4788-88c7-a8865faa47ac admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.844976 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.845656 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.845945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.846221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.846959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.848758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.861339 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1394/2742] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:31:07.865493 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.876646 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74794588-97c8-43fb-ba78-76a403ce6831 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.881518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.882185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.882501 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.882877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.883651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.885298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae86520-3c2e-425d-affb-ac689ea38c37 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:07.896950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1349/2743] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:31:07.901221 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:07.912341 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bb0a2-29c0-4780-9cb4-343328f86210 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:07.917282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:07.918407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:07.918669 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:07.918977 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] PATH_INFO: `/v3/projects/58ef2b1d68a745e1b1ff920b6f4a41ba/users/f29a30e19eaf4f09aa39826b740e86a8/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:07.935908 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ef2b1d68a745e1b1ff920b6f4a41ba, user_id=f29a30e19eaf4f09aa39826b740e86a8, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:07.938555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:08.037984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1395/2744] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:31:07 2022] PUT /identity/v3/projects/58ef2b1d68a745e1b1ff920b6f4a41ba/users/f29a30e19eaf4f09aa39826b740e86a8/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:31:08.049425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:08.049748 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:08.050026 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:08.078841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] MFA Rules not processed for user `f29a30e19eaf4f09aa39826b740e86a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:31:08.080147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:08.147012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1350/2745] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:31:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:31:08.158564 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:08.160309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:08.183698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:08.221892 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c650b-bfac-4c76-8cc8-633ca034af51 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:08.225583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:08.229917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:08.229917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:08.229917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:08.235561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:08.237001 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b87dc5-1685-44af-a36a-ac9068d01429 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:08.246141 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1396/2746] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:31:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:31:12.223258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:12.243570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:12.289714 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab48ae-282a-45df-a551-5e771cb0c5f1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:12.294658 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:12.295556 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:12.295843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:12.296118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] PATH_INFO: `/v3/users/f29a30e19eaf4f09aa39826b740e86a8` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:12.297899 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29a30e19eaf4f09aa39826b740e86a8)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:12.299969 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:12.513436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29a30e19eaf4f09aa39826b740e86a8'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:12.514112 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29a30e19eaf4f09aa39826b740e86a8'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:12.527418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29a30e19eaf4f09aa39826b740e86a8'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:12.535291 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29a30e19eaf4f09aa39826b740e86a8'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:12.602584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29a30e19eaf4f09aa39826b740e86a8'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:12.609122 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1351/2747] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:31:12 2022] DELETE /identity/v3/users/f29a30e19eaf4f09aa39826b740e86a8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:31:12.618191 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:12.618717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:12.618827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:12.636494 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:31:12.637720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:12.706186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1397/2748] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:31:12.715977 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:12.717428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:12.784923 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd826b9-47e9-4e5d-a9e9-c2566fde2a90 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:12.793855 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:12.795450 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:12.796175 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:12.796769 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:12.806577 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:12.808071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b159c-6b4c-4d42-85f7-9f140a351f3b service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:12.819421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1352/2749] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:31:13.548074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:13.549580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:13.561779 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f653b26-cc47-4d20-ba22-75ecf1e329d6 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:13.567583 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:13.568458 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:13.568739 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:13.569013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] PATH_INFO: `/v3/projects/58ef2b1d68a745e1b1ff920b6f4a41ba` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:13.570219 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ef2b1d68a745e1b1ff920b6f4a41ba)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:13.572169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:13.718206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ef2b1d68a745e1b1ff920b6f4a41ba', 'request_id': 'req-af948eea-2054-4c65-892e-3ce4aaea32f1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:13.724389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ef2b1d68a745e1b1ff920b6f4a41ba', 'request_id': 'req-af948eea-2054-4c65-892e-3ce4aaea32f1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:13.791535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ef2b1d68a745e1b1ff920b6f4a41ba', 'request_id': 'req-af948eea-2054-4c65-892e-3ce4aaea32f1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:13.803929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] The token cache is being invalidate because project 58ef2b1d68a745e1b1ff920b6f4a41ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:31:13.804278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:13.804915 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:31:13.806483 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1398/2750] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:31:13 2022] DELETE /identity/v3/projects/58ef2b1d68a745e1b1ff920b6f4a41ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:31:22.813758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:23.801182 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:32:10.992326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:10.994985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:10.994985 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:11.011567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] MFA Rules not processed for user `796d04c1e8a348d9ba3832cb0ede80f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:32:11.012787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:11.075391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1353/2751] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:32:11.113352 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:32:11.114793 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:11.173792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af948eea-2054-4c65-892e-3ce4aaea32f1 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:32:11.177709 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbfd6e827b9e49949cd6543edc89bbaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:32:11.178366 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:11.178647 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:11.178919 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:11.182016 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:11.255009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:32:11.256487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:32:11.267797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1399/2752] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:32:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:32:11.929339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:32:11.931308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:11.989782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8d208-1489-4f6d-9fdd-e1f4755810a0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:32:11.993726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:32:11.994383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:11.994669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:11.994950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:11.996777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:12.011288 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:32:12.012778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aa65d-b69b-4fff-ba5d-e44961143c1c service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:32:12.025216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1354/2753] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:32:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:32:21.262778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=54 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:32:22.019819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=51 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:32:32.512527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:32:32.513971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:32.571253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2eafb-e1c3-4b71-8f6d-defec3e7a57b service glance] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:32:32.574831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:32:32.575508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:32.575787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:32.576061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:32.577808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:32.642534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:32:32.644008 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:32:32.653523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1400/2754] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:32:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:32:42.649351 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=48 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:32:45.191201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:45.191905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:45.191905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:45.209422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] MFA Rules not processed for user `c466157931bd41baabb01abbec492b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:32:45.210634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:45.218233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1355/2755] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:32:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:32:49.019227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:32:49.021017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:49.034482 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628095-6f19-434d-b83a-2990a62f3363 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:32:49.038682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:32:49.039688 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:32:49.039972 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:32:49.040249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:32:49.042842 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:32:49.054337 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:32:49.056078 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:32:49.066162 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1401/2756] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:32:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:32:55.204238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:32:59.061301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=6 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:33:06.291436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:33:06.303726 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dcc00-1421-40b3-a14d-6a0c4bf962c1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:33:06.307601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:33:06.308247 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:33:06.308527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:33:06.308799 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:33:06.310612 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:33:06.379092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:33:06.380576 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8052d44-6fba-44f8-b010-da76e8965c76 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:33:06.391053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1356/2757] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:33:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:33:16.385891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=30 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:33:56.241492 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:33:56.243067 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:33:56.259881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8fff-c7b5-4e1e-b598-2abadc0f4234 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:33:56.265625 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:33:56.266282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:33:56.266585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:33:56.266846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:33:56.268691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:33:56.284232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:33:56.285933 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:33:56.300304 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1402/2758] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:33:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:34:00.862293 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:34:00.862810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:34:00.862810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:34:00.881096 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] MFA Rules not processed for user `796d04c1e8a348d9ba3832cb0ede80f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:34:00.883760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:34:00.889186 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1357/2759] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:34:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:34:01.618082 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:34:01.619510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:34:01.691875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74663374-eaea-430b-a69a-62ac7ba4410f service nova] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:34:01.697080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:34:01.697734 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:34:01.698018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:34:01.698295 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:34:01.700075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:34:01.712856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:34:01.714436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:34:01.724681 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1403/2760] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:34:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:34:10.875693 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:34:11.720391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=38 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:34:13.113596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:34:13.115645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:34:13.131791 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0a27d-d42b-41cb-82c2-cd3ace95d712 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:34:13.138504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:34:13.138504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:34:13.138504 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:34:13.138865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:34:13.140991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:34:13.153400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:34:13.155749 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:34:13.166523 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1358/2761] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:34:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:34:23.162098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=6 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:20.405949 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:20.480874 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab6b22-f30c-4f2b-8db3-c7e29900188a service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:20.489476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:20.490076 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:20.490367 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:20.490679 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:20.492626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:20.532484 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:20.534155 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:20.546119 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1404/2762] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:25.835851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:25.856020 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b586903-f074-4961-bf7e-0aa3e7175ab8 service nova] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:25.860774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:25.861418 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:25.861698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:25.861981 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:25.863777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:25.948421 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:25.949873 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:25.969993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1359/2763] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:35:29.546195 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:29.558330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4ffbb-8a1e-41fb-a61c-00f303bb1ad2 service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:29.564282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:29.564937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:29.565223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:29.565500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:29.567749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:29.581175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:29.583272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:29.594531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1405/2764] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:32.165939 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:32.202357 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3302fe-824d-4f52-aeb9-fe3219cbb9d4 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:32.209461 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c101547390374884b75d95b637267e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:32.210780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:32.210780 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:32.211196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:32.213818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:32.227410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:32.229605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:32.244903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1360/2765] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:35:33.451056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:33.452540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:33.468307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268810-95f8-4821-b1d9-9632aa275191 service placement] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:33.473125 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:33.474003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:33.474289 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:33.474566 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] PATH_INFO: `/v3/users/c466157931bd41baabb01abbec492b4f` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:33.475773 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c466157931bd41baabb01abbec492b4f)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:33.477837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:33.664751 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c466157931bd41baabb01abbec492b4f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:33.672463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c466157931bd41baabb01abbec492b4f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:33.685543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c466157931bd41baabb01abbec492b4f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:33.686242 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c466157931bd41baabb01abbec492b4f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:33.758639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c466157931bd41baabb01abbec492b4f'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:33.766573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1406/2766] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:35:33 2022] DELETE /identity/v3/users/c466157931bd41baabb01abbec492b4f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:35:34.028544 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:34.042482 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ed21f-7ecb-4893-a97e-ac68870942f9 service nova] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:34.104072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:34.104937 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:34.105222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:34.105498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] PATH_INFO: `/v3/projects/f3904fb55e2e463fb548333d97601ce5` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:34.107024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3904fb55e2e463fb548333d97601ce5)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:34.108886 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:34.247068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:34.263455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3904fb55e2e463fb548333d97601ce5', 'request_id': 'req-5ca1a1f6-c0b0-4a85-a20c-83f924554599'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.269626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3904fb55e2e463fb548333d97601ce5', 'request_id': 'req-5ca1a1f6-c0b0-4a85-a20c-83f924554599'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.276409 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3904fb55e2e463fb548333d97601ce5', 'request_id': 'req-5ca1a1f6-c0b0-4a85-a20c-83f924554599'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.375677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] The token cache is being invalidate because project f3904fb55e2e463fb548333d97601ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:35:34.376236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.377113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.378696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1361/2767] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:35:34 2022] DELETE /identity/v3/projects/f3904fb55e2e463fb548333d97601ce5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:35:34.382942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:34.384396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:34.398536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e12-d05e-47a7-85a7-a75d23e8cb63 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:34.438131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:34.457461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:34.458374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:34.458655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:34.458936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] PATH_INFO: `/v3/users/e29df8c17adb49a28acec3d796dfb1c1` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:34.460225 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29df8c17adb49a28acec3d796dfb1c1)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:34.462307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:34.629830 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29df8c17adb49a28acec3d796dfb1c1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.636994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29df8c17adb49a28acec3d796dfb1c1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.649232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29df8c17adb49a28acec3d796dfb1c1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.649924 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29df8c17adb49a28acec3d796dfb1c1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.718510 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-6ea4c93a-557d-47ec-8a57-f357e026935d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29df8c17adb49a28acec3d796dfb1c1'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:34.725216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1407/2768] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:35:34 2022] DELETE /identity/v3/users/e29df8c17adb49a28acec3d796dfb1c1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:35:34.995338 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:34.999746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:35.012153 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1a1f6-c0b0-4a85-a20c-83f924554599 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:35.081690 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:35.082572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:35.082851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:35.083150 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] PATH_INFO: `/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:35.084386 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6e12d349c24bd885349dbd2e4cc91d)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:35.086433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:35.197906 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6e12d349c24bd885349dbd2e4cc91d', 'request_id': 'req-c93f4205-55c0-40ad-b589-d0e774cfd1cc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:35.203333 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6e12d349c24bd885349dbd2e4cc91d', 'request_id': 'req-c93f4205-55c0-40ad-b589-d0e774cfd1cc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:35.209823 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6e12d349c24bd885349dbd2e4cc91d', 'request_id': 'req-c93f4205-55c0-40ad-b589-d0e774cfd1cc'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:35.284113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] The token cache is being invalidate because project 8c6e12d349c24bd885349dbd2e4cc91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:35:35.284431 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:35.285059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:35:35.286601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1362/2769] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:35:34 2022] DELETE /identity/v3/projects/8c6e12d349c24bd885349dbd2e4cc91d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:35:44.721578 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:45.280669 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:35:53.247523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:53.248059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:53.248059 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:53.268107 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] MFA Rules not processed for user `796d04c1e8a348d9ba3832cb0ede80f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:35:53.269569 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:53.324774 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1408/2770] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:35:53.636055 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:35:53.637595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:53.718691 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f4205-55c0-40ad-b589-d0e774cfd1cc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:35:53.723224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:35:53.727593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:35:53.727593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:35:53.727593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:35:53.727593 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:35:53.736579 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:35:53.738415 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:35:53.746952 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1363/2771] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:35:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:36:03.317824 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=25 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:36:03.743203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=35 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:15.961459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:15.963724 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:16.056056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802461a-41e0-45c3-8695-29c2070dfc5a None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:16.060462 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:16.061189 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:16.061693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:16.062163 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:16.064279 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:16.166211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:16.167605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:16.180627 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1409/2772] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:25.785344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:25.794680 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d461d-9521-4bbc-a5c2-81b15dbd2178 service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:25.798235 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:25.798865 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:25.799236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:25.799561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:25.801766 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:25.815613 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:25.817082 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:25.828959 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1364/2773] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:26.174180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=68 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:35.823425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=6 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:38.217335 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:38.218717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:38.231533 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ebfd-1138-455c-a63d-a74ee5689ad1 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:38.236002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:38.236657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:38.236965 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:38.237248 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:38.238999 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:38.302218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:38.303696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:38.313780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1410/2774] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:41.572170 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:41.573677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:41.647531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285774c7-5426-4ef3-a300-96bc4d65ce9e service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:41.652598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:41.653250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:41.653538 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:41.653810 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:41.655596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:41.668592 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:41.670171 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:41.681208 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1365/2775] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:41.712770 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:41.714320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:41.729151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc568c-b2cd-4718-be67-786361fbc7fa service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:41.733829 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:41.734730 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:41.735010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:41.735412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] PATH_INFO: `/v3/users/796d04c1e8a348d9ba3832cb0ede80f3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:41.736612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796d04c1e8a348d9ba3832cb0ede80f3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:41.738534 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:41.923990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796d04c1e8a348d9ba3832cb0ede80f3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:41.930683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796d04c1e8a348d9ba3832cb0ede80f3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:41.942798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796d04c1e8a348d9ba3832cb0ede80f3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:41.943495 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796d04c1e8a348d9ba3832cb0ede80f3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.030041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-e9156f4b-351e-4bd6-b334-58268e940787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796d04c1e8a348d9ba3832cb0ede80f3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.036918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1411/2776] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:41 2022] DELETE /identity/v3/users/796d04c1e8a348d9ba3832cb0ede80f3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:42.415233 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:42.427264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972576-8670-449c-9d6c-71924aa7fdfd service placement] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:42.493275 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:42.494281 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:42.494406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:42.494682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] PATH_INFO: `/v3/projects/9a3aebe2cb6342ce80d1979e6c330815` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:42.495918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3aebe2cb6342ce80d1979e6c330815)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:42.497872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:42.600535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3aebe2cb6342ce80d1979e6c330815', 'request_id': 'req-8d0aeb04-f9c7-401f-b58b-f4172100467e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.606195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3aebe2cb6342ce80d1979e6c330815', 'request_id': 'req-8d0aeb04-f9c7-401f-b58b-f4172100467e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.613341 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3aebe2cb6342ce80d1979e6c330815', 'request_id': 'req-8d0aeb04-f9c7-401f-b58b-f4172100467e'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.681455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] The token cache is being invalidate because project 9a3aebe2cb6342ce80d1979e6c330815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:42.681676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.682283 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8d0aeb04-f9c7-401f-b58b-f4172100467e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:42.683877 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1366/2777] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:42 2022] DELETE /identity/v3/projects/9a3aebe2cb6342ce80d1979e6c330815 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:42.689428 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:42.689720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:42.689997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:42.746409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:42.747636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:42.776986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:42.805737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1412/2778] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:42.813745 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:42.814037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:42.814359 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:42.833374 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:42.834590 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:42.842746 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1367/2779] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:42.847685 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:42.857597 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8aedc4-1401-43b3-95f9-637a39efc4d3 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:42.863073 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:42.863790 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:42.864280 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:42.864722 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:42.865755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:37:42.866064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:42.867691 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:42.879272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1413/2780] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:37:42.883517 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:42.893400 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f776b93-a2a1-46de-a554-bc4165f39ffe None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:42.898017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:42.898660 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:42.898945 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:42.899248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:42.899941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:42.901832 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:42.912326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:43.003645 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1368/2781] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:42 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:43.007952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.018608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23de8d-c452-4706-8e7c-430842b19c9d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.073794 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.074468 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.074755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.075032 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.075797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.077758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.190641 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1414/2782] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:43 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:37:43.194808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.206017 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c74c5-8798-4395-b5b6-c90520195d96 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.211900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.212565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.212852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.213131 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.213864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.215507 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.226378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1369/2783] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:43.230516 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.240670 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0a90-6948-42a1-bbba-120da2c02daa admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.245847 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.246518 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.246805 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.247083 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.247850 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.249459 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8ab28-a153-439a-a31d-7887bde9b719 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.260399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1415/2784] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:43.264534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.275216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2536e72-a492-4c0d-9f96-555cb18b34c1 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.280809 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.281688 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.281972 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.282250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] PATH_INFO: `/v3/projects/19a11bc7a8064d1ab89003ed64844cbd/users/70ca0fe0cea7436eb9fb94fdb2fc3d2c/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.296808 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a11bc7a8064d1ab89003ed64844cbd, user_id=70ca0fe0cea7436eb9fb94fdb2fc3d2c, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.299478 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.379187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1370/2785] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:43 2022] PUT /identity/v3/projects/19a11bc7a8064d1ab89003ed64844cbd/users/70ca0fe0cea7436eb9fb94fdb2fc3d2c/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:43.390253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.390548 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.391466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.418186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] MFA Rules not processed for user `70ca0fe0cea7436eb9fb94fdb2fc3d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:43.419606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:43.486164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1416/2786] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:37:43.490596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.556986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014d80-0840-4fd1-b853-8a8a26c67e30 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.561682 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.562339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.563107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.563539 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.564328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.566297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.712185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1371/2787] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:43 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:43.716175 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.727300 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b96264-620b-47ed-a7ea-007d785eebc0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.757057 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:43.783704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.784373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.784657 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.784936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.785614 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.787562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.892206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1417/2788] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:43 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:43.895912 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.906190 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0333-0437-4842-b2bb-168ace5f3143 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.910994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.911677 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.911964 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.912238 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.912956 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.914584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:43.925635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1372/2789] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:43.929649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:43.940586 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda024cf-11cb-4133-840a-99b715ff7a70 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:43.945265 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:43.946134 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:43.946442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:43.946723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] PATH_INFO: `/v3/projects/e7d09f4d91a94086972c0cccad47cf22/users/41c9c100e4e54c5b8457e306ce7970e4/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:43.961504 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d09f4d91a94086972c0cccad47cf22, user_id=41c9c100e4e54c5b8457e306ce7970e4, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:43.964198 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.036497 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1418/2790] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:43 2022] PUT /identity/v3/projects/e7d09f4d91a94086972c0cccad47cf22/users/41c9c100e4e54c5b8457e306ce7970e4/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:44.040789 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:44.050933 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777aa32-4415-4906-bf53-714b4be11514 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:44.107009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:44.107701 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.107987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.108264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.108986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:44.110606 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dde35-98c1-4bf1-b148-8de770d8a326 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.120234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1373/2791] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:44.124384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:44.135430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655910b-5a72-4d7c-9fd2-23177bb06b17 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:44.140048 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:44.141576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.141869 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.142147 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] PATH_INFO: `/v3/projects/e7d09f4d91a94086972c0cccad47cf22/users/41c9c100e4e54c5b8457e306ce7970e4/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.144224 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d09f4d91a94086972c0cccad47cf22, user_id=41c9c100e4e54c5b8457e306ce7970e4, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:44.146885 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.157061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1419/2792] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:44 2022] PUT /identity/v3/projects/e7d09f4d91a94086972c0cccad47cf22/users/41c9c100e4e54c5b8457e306ce7970e4/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:44.166920 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.167250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.167533 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.194666 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] MFA Rules not processed for user `41c9c100e4e54c5b8457e306ce7970e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:44.195926 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:44.200845 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:44.268059 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1374/2793] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:44.279487 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:44.281169 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:44.338095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125fda-9874-4a56-a0e5-26d0bd620454 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:44.341753 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:44.342398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.342683 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.342959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.352802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:44.355738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.364988 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1420/2794] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:44.545595 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:44.555942 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6aa147-d9ae-4967-aed5-15b4d2ea1c46 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:44.559527 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:44.560165 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:44.560457 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:44.560737 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:44.618520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:44.619994 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec4bce-d075-46f1-983a-c6eafdf5fe1b service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:44.628989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1375/2795] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:45.218414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:45.283355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dc661-e134-4c73-aad8-743c82cf91f5 service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:45.287983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:45.289301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:45.289301 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:45.289523 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] PATH_INFO: `/v3/users/70ca0fe0cea7436eb9fb94fdb2fc3d2c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:45.290604 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ca0fe0cea7436eb9fb94fdb2fc3d2c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:45.292598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:45.379441 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:45.512164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ca0fe0cea7436eb9fb94fdb2fc3d2c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:45.519994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ca0fe0cea7436eb9fb94fdb2fc3d2c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:45.531591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ca0fe0cea7436eb9fb94fdb2fc3d2c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:45.532348 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ca0fe0cea7436eb9fb94fdb2fc3d2c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:45.600275 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ca0fe0cea7436eb9fb94fdb2fc3d2c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:45.606994 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1421/2796] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:45 2022] DELETE /identity/v3/users/70ca0fe0cea7436eb9fb94fdb2fc3d2c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:45.611768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:45.612391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:45.612391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:45.630818 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:45.632048 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:45.690814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1376/2797] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:45.700545 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:45.701959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:45.759776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb06a1-b9eb-458a-9748-95d0dc706942 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:45.763501 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:45.764088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:45.764377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:45.764662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:45.774422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:45.775881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:45.785587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1422/2798] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:46.151896 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:46.162487 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:46.165413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de848ab-2dba-4f71-83ff-a77224d38c78 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:46.170016 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:46.170890 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:46.171197 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:46.171475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] PATH_INFO: `/v3/projects/19a11bc7a8064d1ab89003ed64844cbd` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:46.172684 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a11bc7a8064d1ab89003ed64844cbd)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:46.175384 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:46.251226 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:46.260218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a11bc7a8064d1ab89003ed64844cbd', 'request_id': 'req-b24bafe4-bada-42a6-99c7-b0aac5c07b20'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.265657 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a11bc7a8064d1ab89003ed64844cbd', 'request_id': 'req-b24bafe4-bada-42a6-99c7-b0aac5c07b20'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.272014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a11bc7a8064d1ab89003ed64844cbd', 'request_id': 'req-b24bafe4-bada-42a6-99c7-b0aac5c07b20'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.321177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] The token cache is being invalidate because project 19a11bc7a8064d1ab89003ed64844cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:46.321626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.322699 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.324502 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1377/2799] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:46 2022] DELETE /identity/v3/projects/19a11bc7a8064d1ab89003ed64844cbd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:46.328537 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:46.338432 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db869fe-d8e6-43f1-9b61-31ddbfc6d97c service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:46.396828 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:46.397704 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:46.397983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:46.398252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] PATH_INFO: `/v3/users/41c9c100e4e54c5b8457e306ce7970e4` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:46.399496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c9c100e4e54c5b8457e306ce7970e4)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:46.401398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:46.591372 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c9c100e4e54c5b8457e306ce7970e4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.593364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:46.598776 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c9c100e4e54c5b8457e306ce7970e4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.611389 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c9c100e4e54c5b8457e306ce7970e4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.615291 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c9c100e4e54c5b8457e306ce7970e4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.697473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-07f7cfaf-476e-4194-84d4-a79b80512f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c9c100e4e54c5b8457e306ce7970e4'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:46.703906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1423/2800] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:46 2022] DELETE /identity/v3/users/41c9c100e4e54c5b8457e306ce7970e4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:47.105117 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.106584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:47.120841 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bafe4-bada-42a6-99c7-b0aac5c07b20 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.183713 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.184752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.185046 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.185327 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] PATH_INFO: `/v3/projects/e7d09f4d91a94086972c0cccad47cf22` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.186535 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d09f4d91a94086972c0cccad47cf22)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.188600 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.327834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d09f4d91a94086972c0cccad47cf22', 'request_id': 'req-3a314879-7d88-42b8-a49f-5778139885b4'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:47.334355 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d09f4d91a94086972c0cccad47cf22', 'request_id': 'req-3a314879-7d88-42b8-a49f-5778139885b4'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:47.340676 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d09f4d91a94086972c0cccad47cf22', 'request_id': 'req-3a314879-7d88-42b8-a49f-5778139885b4'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:47.412021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] The token cache is being invalidate because project e7d09f4d91a94086972c0cccad47cf22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:47.412259 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:47.412957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-3a314879-7d88-42b8-a49f-5778139885b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:47.414529 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1378/2801] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:47 2022] DELETE /identity/v3/projects/e7d09f4d91a94086972c0cccad47cf22 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:47.419383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.419872 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.420012 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.438122 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:47.439496 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:47.497686 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1424/2802] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:47.505605 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.505902 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.506187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.525752 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:47.527067 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:47.537907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1379/2803] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:47.542099 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.552375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88e25a-dcfe-43b5-a638-1f410e12eacd None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.557131 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.557787 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.558066 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.558347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.559069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:37:47.559429 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.561064 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.580755 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1425/2804] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:37:47.585785 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.597760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1271cb-da27-449e-b5b9-8b73eeda05a7 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.603573 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.604647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.605222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.605542 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.606224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.608557 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.741189 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1380/2805] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:47 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:47.745346 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.756474 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d20a-f826-4543-a6dc-7bda7f85c26d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.816795 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.817499 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.817766 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.818019 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.818708 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.820715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.913133 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1426/2806] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:47 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:37:47.917183 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.927903 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0e092-18d9-4fa3-81fb-1f3f6ed199b8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.932531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.933185 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.933468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.933747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.934468 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.936144 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.948433 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1381/2807] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:47.952380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.963422 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb057a6-0cf0-4284-9004-b9ab653c577d admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:47.968053 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:47.968701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:47.968982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:47.969253 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:47.970002 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:47.971825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23918c5-a539-47b6-8673-0f7ad71377c7 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:47.982407 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1427/2808] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:47.986798 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:47.996905 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c9d18-1071-4499-b943-32873ce7595b admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.002307 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.003216 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.003500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.003775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] PATH_INFO: `/v3/projects/ce4328ae12a846d8a5c02580048a98bb/users/77c3fc4287b24371ba813ea7859ebad9/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.015900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:48.019887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4328ae12a846d8a5c02580048a98bb, user_id=77c3fc4287b24371ba813ea7859ebad9, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.022598 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.091834 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1382/2809] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:47 2022] PUT /identity/v3/projects/ce4328ae12a846d8a5c02580048a98bb/users/77c3fc4287b24371ba813ea7859ebad9/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:37:48.103309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758001-c805-4bde-8957-ecb7df4300af None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.103602 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758001-c805-4bde-8957-ecb7df4300af None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.104130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758001-c805-4bde-8957-ecb7df4300af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.132286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-68758001-c805-4bde-8957-ecb7df4300af None None] MFA Rules not processed for user `77c3fc4287b24371ba813ea7859ebad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:48.133506 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-68758001-c805-4bde-8957-ecb7df4300af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:48.153206 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:48.199983 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1428/2810] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:48.205136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.270405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2387-a207-462a-a080-451f90cf445c admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.275098 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.275786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.276072 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.276348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.277070 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.279034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.355893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1383/2811] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:48 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:48.360088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68758001-c805-4bde-8957-ecb7df4300af None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.371466 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68758001-c805-4bde-8957-ecb7df4300af None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.435640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.436313 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.436595 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.436866 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.437551 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.440557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.509266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1429/2812] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:48 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:48.513236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.524111 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c2d2c-9665-44bc-8968-6f0ce89d88a4 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.528689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.529345 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.529633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.529909 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.530671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.532782 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.543548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1384/2813] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:37:48.549556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.559591 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d6243-1bd5-45af-9b73-2873be5744a5 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.564299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.565179 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.565464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.565741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] PATH_INFO: `/v3/projects/4af91cabd93340109acac36325788b09/users/44c0c392fc0c45dc9dbc534de1b847b3/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.580330 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af91cabd93340109acac36325788b09, user_id=44c0c392fc0c45dc9dbc534de1b847b3, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.582991 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.641952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1430/2814] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:48 2022] PUT /identity/v3/projects/4af91cabd93340109acac36325788b09/users/44c0c392fc0c45dc9dbc534de1b847b3/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:48.646546 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.656597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309a93-4f8b-4bdc-a6fd-036bd819712f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.713213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.713914 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.714201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.714483 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.715237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.716843 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d01059-f505-4476-ba58-80c37696547f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.727422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1385/2815] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:48.731918 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.743581 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c95603-0d92-4cc2-98c3-0f712216c40e admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.748639 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.749520 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.749807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.750215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] PATH_INFO: `/v3/projects/4af91cabd93340109acac36325788b09/users/44c0c392fc0c45dc9dbc534de1b847b3/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.752540 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af91cabd93340109acac36325788b09, user_id=44c0c392fc0c45dc9dbc534de1b847b3, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.755282 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.765810 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1431/2816] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:48 2022] PUT /identity/v3/projects/4af91cabd93340109acac36325788b09/users/44c0c392fc0c45dc9dbc534de1b847b3/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:48.775842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.776071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.776350 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.805352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] MFA Rules not processed for user `44c0c392fc0c45dc9dbc534de1b847b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:48.806563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:48.854817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:48.875031 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1386/2817] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:48.886944 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:48.888673 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:48.947374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fac7-9359-4474-9c8a-9315f3c38068 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:48.950963 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:48.951636 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:48.951917 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:48.952188 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:48.962258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:48.963742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:48.973818 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1432/2818] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:49.094443 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:49.096874 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:49.109113 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fe0c-d5a8-4349-9c7f-7cf3955f7f8d None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:49.113403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:49.114050 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:49.114342 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:49.114622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:49.172129 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:49.173583 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240d90-938c-4f47-aecf-5ae8db39254e service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:49.182965 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1387/2819] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:54.021149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:54.040851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:54.085640 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b8479-531f-4b0d-9c9e-1d067e0f6a5d service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:54.090237 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:54.091183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:54.091473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:54.091747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] PATH_INFO: `/v3/users/77c3fc4287b24371ba813ea7859ebad9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:54.093074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c3fc4287b24371ba813ea7859ebad9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:54.095052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:54.253930 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c3fc4287b24371ba813ea7859ebad9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:54.260780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c3fc4287b24371ba813ea7859ebad9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:54.273414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c3fc4287b24371ba813ea7859ebad9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:54.274180 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c3fc4287b24371ba813ea7859ebad9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:54.339025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c3fc4287b24371ba813ea7859ebad9'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:54.345263 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1433/2820] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:54 2022] DELETE /identity/v3/users/77c3fc4287b24371ba813ea7859ebad9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:54.350898 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:54.351426 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:54.351548 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:54.369585 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:54.370800 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:54.432000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1388/2821] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:54.442765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:54.444192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:54.500645 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e93b4-e487-4a82-8eda-87b05986dce0 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:54.505060 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:54.505653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:54.505936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:54.506213 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:54.515269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:54.516698 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:54.526827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1434/2822] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:37:54.893349 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:54.894819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:54.907151 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe685e-2988-4932-8e6e-cb41361464ff None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:54.912161 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:54.913043 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:54.913330 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:54.913607 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] PATH_INFO: `/v3/projects/ce4328ae12a846d8a5c02580048a98bb` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:54.914803 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4328ae12a846d8a5c02580048a98bb)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:54.916754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:55.040631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4328ae12a846d8a5c02580048a98bb', 'request_id': 'req-f0e637f0-eced-4649-8694-615c6503925a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.046071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4328ae12a846d8a5c02580048a98bb', 'request_id': 'req-f0e637f0-eced-4649-8694-615c6503925a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.052911 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4328ae12a846d8a5c02580048a98bb', 'request_id': 'req-f0e637f0-eced-4649-8694-615c6503925a'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.142567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] The token cache is being invalidate because project ce4328ae12a846d8a5c02580048a98bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:55.142856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.143650 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.145217 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1389/2823] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:54 2022] DELETE /identity/v3/projects/ce4328ae12a846d8a5c02580048a98bb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:55.149325 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:55.159446 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3291728-47b8-4779-b4ba-8c54dd662614 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:55.199349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:55.217151 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:55.218025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:55.218308 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:55.218584 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] PATH_INFO: `/v3/users/44c0c392fc0c45dc9dbc534de1b847b3` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:55.219797 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c0c392fc0c45dc9dbc534de1b847b3)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:55.222322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:55.409528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c0c392fc0c45dc9dbc534de1b847b3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.416326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c0c392fc0c45dc9dbc534de1b847b3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.429227 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c0c392fc0c45dc9dbc534de1b847b3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.429926 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c0c392fc0c45dc9dbc534de1b847b3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.520293 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-de638396-3877-4ee9-ad30-adad98bab459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c0c392fc0c45dc9dbc534de1b847b3'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:55.526521 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1435/2824] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:55 2022] DELETE /identity/v3/users/44c0c392fc0c45dc9dbc534de1b847b3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:55.952459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:55.953963 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:55.968448 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e637f0-eced-4649-8694-615c6503925a admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:55.974162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:56.028210 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.029126 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.029410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.029689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] PATH_INFO: `/v3/projects/4af91cabd93340109acac36325788b09` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.030935 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af91cabd93340109acac36325788b09)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.032876 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.161469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af91cabd93340109acac36325788b09', 'request_id': 'req-12694050-7b2d-4d65-b792-33a2170c82d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:56.167385 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af91cabd93340109acac36325788b09', 'request_id': 'req-12694050-7b2d-4d65-b792-33a2170c82d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:56.173520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af91cabd93340109acac36325788b09', 'request_id': 'req-12694050-7b2d-4d65-b792-33a2170c82d5'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:56.263068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] The token cache is being invalidate because project 4af91cabd93340109acac36325788b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:56.263398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:56.264008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-12694050-7b2d-4d65-b792-33a2170c82d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:56.265561 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1390/2825] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:55 2022] DELETE /identity/v3/projects/4af91cabd93340109acac36325788b09 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:56.270701 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.271218 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.271350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.290968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:56.292186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:56.351316 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1436/2826] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:56.359481 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.364642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.364642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.379966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:56.381220 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:56.389480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1391/2827] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:56.394649 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.409827 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523868c3-d8ef-465d-9957-03a1bea84745 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.411216 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.411911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.412200 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.412478 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.413232 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:37:56.413544 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.415207 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.427525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1437/2828] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:37:56.431888 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.441405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817e1f-cc3d-4864-98d7-9b4da41f52d9 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.446009 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.446689 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.446973 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.447274 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.447982 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.449871 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.541774 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1392/2829] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:56 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:56.546062 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.556419 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e23d-cb95-41bb-9af0-fe418cad74cd admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.601705 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:56.614563 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.615273 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.615568 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.615845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.616525 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.618464 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.687493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1438/2830] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:56 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:56.692339 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.702854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70104-bbf0-4d8a-8ef8-eb8b32294b13 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.708651 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.709322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.709641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.710039 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.710778 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.712428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.723988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1393/2831] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:37:56.728046 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.739782 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4abc-f704-4be9-a3c0-4b32d7896006 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.744825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.745555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.745853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.746149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.746931 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.748577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4d54d-4a51-4eb9-afb3-9d10db61b2b4 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.758856 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1439/2832] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:56.763254 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:56.773534 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88518-4707-4c75-a8f5-655fcf1dcc41 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:56.778411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:56.779352 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.779640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.779919 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] PATH_INFO: `/v3/projects/9bd5e0b2c8034d7ba2c63611f9306d1a/users/b3fd614e859c443c91fa9ba4c0679f8f/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.795407 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd5e0b2c8034d7ba2c63611f9306d1a, user_id=b3fd614e859c443c91fa9ba4c0679f8f, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:56.798002 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:56.853828 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1394/2833] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:56 2022] PUT /identity/v3/projects/9bd5e0b2c8034d7ba2c63611f9306d1a/users/b3fd614e859c443c91fa9ba4c0679f8f/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:56.918780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:56.919069 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:56.919386 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:56.945528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] MFA Rules not processed for user `b3fd614e859c443c91fa9ba4c0679f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:56.946742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:57.010234 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1440/2834] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:57.014851 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.054335 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:57.076980 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962f51b-5abd-44c7-858e-d23724dfbfe3 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.081760 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.082420 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.082706 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.082983 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.083711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.085642 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.187927 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1395/2835] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:57 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:57.192845 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.203347 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10427dfa-81a3-4af3-93ae-2ffc25939aad None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.258897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.259598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.259882 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.260160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.260836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.262737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.333876 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1441/2836] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:37:57 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:57.337829 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.350454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c752-e633-4255-883b-086e6a516eb8 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.354698 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.355467 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.355735 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.356014 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.356740 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.358367 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.370516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1396/2837] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:57.374687 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.386894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741237e-c280-4921-8b0b-15d39909c8ce admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.391570 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.392448 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.392737 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.393081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] PATH_INFO: `/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1/users/455fcd73df6e4f9fbcc1337d81e64c88/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.408003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cb5e46b56f4a89b23a14535dcee6c1, user_id=455fcd73df6e4f9fbcc1337d81e64c88, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.410897 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.482577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1442/2838] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:57 2022] PUT /identity/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1/users/455fcd73df6e4f9fbcc1337d81e64c88/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:57.486570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.497618 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ae0f5-60e3-4fd0-a360-73138b83e32b admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.554383 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.555047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.555362 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.555640 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.556358 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.557953 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716b96-3d4e-4940-b610-c2f794330bcf admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.567672 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1397/2839] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:37:57.571851 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.582802 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aa479-b5f2-4962-8299-5a5726acb204 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.587742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.588555 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.588834 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.589127 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] PATH_INFO: `/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1/users/455fcd73df6e4f9fbcc1337d81e64c88/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.591054 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cb5e46b56f4a89b23a14535dcee6c1, user_id=455fcd73df6e4f9fbcc1337d81e64c88, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.593690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.605322 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1443/2840] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:37:57 2022] PUT /identity/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1/users/455fcd73df6e4f9fbcc1337d81e64c88/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:57.615848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.616140 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.616424 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.642010 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] MFA Rules not processed for user `455fcd73df6e4f9fbcc1337d81e64c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:37:57.643351 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:57.710930 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1398/2841] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:37:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:37:57.723875 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:57.725612 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:57.735667 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:57.784971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4eb63-9b3b-483d-a44d-7ac84a648fac admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:57.788619 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:57.789272 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:57.789557 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:57.789837 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:57.800075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:57.801460 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:57.810865 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1444/2842] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:37:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:37:58.692614 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:58.701819 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842ec96-490e-41b2-94b7-a4c5333a8fd1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:58.706497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '455fcd73df6e4f9fbcc1337d81e64c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:58.707264 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:58.709388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:58.709388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:58.709388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:37:58.709388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:58.710474 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:58.722037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1399/2843] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:37:58.726160 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:58.735584 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd455a4-b91b-4efd-9531-3738b850f08d service cinder] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:58.740666 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '455fcd73df6e4f9fbcc1337d81e64c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:58.741340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:58.741626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:58.741911 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:58.742606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:58.744528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:58.824741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1445/2844] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:37:58 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:37:58.828986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:58.839733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37983edf-8827-4281-bfc3-bbf87a8a94cf tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:58.852656 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:37:58.896950 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '455fcd73df6e4f9fbcc1337d81e64c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:58.897543 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:58.897839 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:58.898118 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:58.898842 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:37:58.899237 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:58.900854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:58.912887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1400/2845] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:37:58.948748 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:58.959852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f2f8-d1bd-4bca-8ca7-a4bcaf4bf86e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:58.964387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '455fcd73df6e4f9fbcc1337d81e64c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:58.965238 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:58.965527 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:58.965806 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] PATH_INFO: `/v3/projects/2e2fcd10bcc741078a0732226e815422` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:58.967006 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e2fcd10bcc741078a0732226e815422)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:58.968938 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:37:59.105716 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2fcd10bcc741078a0732226e815422', 'request_id': 'req-83c4286c-c644-4c5a-a530-74f43abf620e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:59.111158 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2fcd10bcc741078a0732226e815422', 'request_id': 'req-83c4286c-c644-4c5a-a530-74f43abf620e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:59.171937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2fcd10bcc741078a0732226e815422', 'request_id': 'req-83c4286c-c644-4c5a-a530-74f43abf620e'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:59.184599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] The token cache is being invalidate because project 2e2fcd10bcc741078a0732226e815422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:37:59.188081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:59.188081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-83c4286c-c644-4c5a-a530-74f43abf620e tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:37:59.188081 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1446/2846] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:37:58 2022] DELETE /identity/v3/projects/2e2fcd10bcc741078a0732226e815422 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:37:59.845492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:37:59.847373 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:37:59.913786 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4edd74-8ecd-4a97-80c6-d3a08ceaade5 tempest-VolumeQuotaClassesTest-638427512 tempest-VolumeQuotaClassesTest-638427512-project-admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:37:59.968584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:37:59.969469 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:37:59.969751 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:37:59.970025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] PATH_INFO: `/v3/users/b3fd614e859c443c91fa9ba4c0679f8f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:37:59.971332 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fd614e859c443c91fa9ba4c0679f8f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:37:59.973776 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:00.140962 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fd614e859c443c91fa9ba4c0679f8f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:00.141622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fd614e859c443c91fa9ba4c0679f8f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:00.152178 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:00.154436 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fd614e859c443c91fa9ba4c0679f8f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:00.161887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fd614e859c443c91fa9ba4c0679f8f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:00.213796 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fd614e859c443c91fa9ba4c0679f8f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:00.220037 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1401/2847] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:37:59 2022] DELETE /identity/v3/users/b3fd614e859c443c91fa9ba4c0679f8f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:00.225479 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:00.225984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:00.225984 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:00.244024 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:00.245229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:00.303906 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1447/2848] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:00.313405 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:00.314961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:00.373904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b928f7-51b7-4e4a-abfc-3d0a2777f1fc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:00.377622 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:00.378298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:00.378589 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:00.378878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:00.389615 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:00.391068 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:00.401512 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1402/2849] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:00.889901 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:00.892594 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:00.906171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688306e9-c676-447a-a4ec-e97d23ac65de None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:00.911562 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:00.912522 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:00.912811 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:00.913090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] PATH_INFO: `/v3/projects/9bd5e0b2c8034d7ba2c63611f9306d1a` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:00.914373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd5e0b2c8034d7ba2c63611f9306d1a)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:00.916509 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:00.923317 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:01.019314 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd5e0b2c8034d7ba2c63611f9306d1a', 'request_id': 'req-62588496-ec32-44e0-8840-372b10498c23'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.025512 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd5e0b2c8034d7ba2c63611f9306d1a', 'request_id': 'req-62588496-ec32-44e0-8840-372b10498c23'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.086309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd5e0b2c8034d7ba2c63611f9306d1a', 'request_id': 'req-62588496-ec32-44e0-8840-372b10498c23'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.097573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] The token cache is being invalidate because project 9bd5e0b2c8034d7ba2c63611f9306d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:01.097857 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.099202 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.100763 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1448/2850] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:00 2022] DELETE /identity/v3/projects/9bd5e0b2c8034d7ba2c63611f9306d1a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:38:01.105092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:01.115109 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92a66-1299-47cd-bada-844d82fd084b service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:01.174222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:01.175143 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:01.175447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:01.175727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] PATH_INFO: `/v3/users/455fcd73df6e4f9fbcc1337d81e64c88` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:01.177403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455fcd73df6e4f9fbcc1337d81e64c88)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:01.179379 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:01.359057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455fcd73df6e4f9fbcc1337d81e64c88'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.359738 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455fcd73df6e4f9fbcc1337d81e64c88'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.372218 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455fcd73df6e4f9fbcc1337d81e64c88'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.379078 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455fcd73df6e4f9fbcc1337d81e64c88'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.470921 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-56ef1c5a-b3b1-4680-8f8d-8c601ebbfa00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455fcd73df6e4f9fbcc1337d81e64c88'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:01.473641 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:01.477872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1403/2851] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:01 2022] DELETE /identity/v3/users/455fcd73df6e4f9fbcc1337d81e64c88 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:01.895756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:01.897269 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:01.913603 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62588496-ec32-44e0-8840-372b10498c23 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:01.970826 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:01.971758 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:01.972041 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:01.972340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] PATH_INFO: `/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:01.974049 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cb5e46b56f4a89b23a14535dcee6c1)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:01.976068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.076920 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'request_id': 'req-787eb26c-0783-440d-9404-5ca69698b347'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:02.082398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'request_id': 'req-787eb26c-0783-440d-9404-5ca69698b347'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:02.150959 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cb5e46b56f4a89b23a14535dcee6c1', 'request_id': 'req-787eb26c-0783-440d-9404-5ca69698b347'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:02.162807 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] The token cache is being invalidate because project f7cb5e46b56f4a89b23a14535dcee6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:02.163094 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:02.163772 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-787eb26c-0783-440d-9404-5ca69698b347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:02.165320 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1449/2852] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:01 2022] DELETE /identity/v3/projects/f7cb5e46b56f4a89b23a14535dcee6c1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:02.170127 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.170601 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.170723 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.189868 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:02.191077 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:02.250136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1404/2853] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:02.259030 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.259349 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.259626 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.278393 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:02.279606 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:02.289061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1450/2854] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:02.293162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.303545 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff0682-d6c9-4a47-9f87-b0dc075380ef None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.308106 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.308757 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.309049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.309329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.310051 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:38:02.310364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.312006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.324107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1405/2855] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:38:02.328377 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.337743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54881b7-4edf-4887-b878-46c94b8190ce None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.342326 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.342966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.343355 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.343630 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.344404 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.346296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.360306 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:02.470935 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1451/2856] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:02.475496 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.486054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa83f1-85b9-493a-ac41-325d0fc41494 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.542388 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.543054 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.543387 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.543667 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.544425 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.546428 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.623814 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1406/2857] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:02.627893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.638881 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd2fc4-2bbe-47ff-b78a-9a15191bd17a admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.643476 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.644124 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.644414 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.644723 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.645580 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.647184 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.657576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1452/2858] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:02.662257 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.672406 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624ca86-f5b6-4960-bc84-201606ef26fb admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.678012 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.678670 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.678955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.679297 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.680030 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.681625 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f376f8-9d8d-4cdc-b831-1ad7330f9b56 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.691575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1407/2859] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:02.696085 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.706174 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d26ec-8c35-48ad-9a47-3953d76ea980 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.710846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.711749 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.712034 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.712310 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] PATH_INFO: `/v3/projects/8fc23de0f1e6440493b295491ec3c871/users/edc73c02c8264783913418d0e6160430/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.727517 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc23de0f1e6440493b295491ec3c871, user_id=edc73c02c8264783913418d0e6160430, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.730461 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:02.795098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1453/2860] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:02 2022] PUT /identity/v3/projects/8fc23de0f1e6440493b295491ec3c871/users/edc73c02c8264783913418d0e6160430/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:02.806364 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.806655 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.806932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.833694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] MFA Rules not processed for user `edc73c02c8264783913418d0e6160430`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:02.834917 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:02.892822 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:02.901824 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1408/2861] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:02.906329 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:02.969074 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3524d-6458-4932-9ae1-4dc568137403 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:02.974780 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:02.975467 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:02.975756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:02.976036 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:02.976727 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:02.978632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.055560 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1454/2862] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:02 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:03.059763 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.070958 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6df4-8d66-4d62-8d59-feec6a531c90 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.127653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.128326 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.128610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.128887 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.129695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.131936 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.202520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1409/2863] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:03 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:03.206655 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.217836 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f911e-947f-4abb-94de-b33168011d2f admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.223838 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.224652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.224937 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.225215 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.225939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.227559 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.238375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1455/2864] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:03.242398 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.252934 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21733998-8f79-445f-8b7f-3bec2d336a94 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.257643 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.258298 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.258580 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.258857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.259610 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.261203 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43252542-ca41-4f75-8fe1-e73536460340 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.271830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1410/2865] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:03.275982 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.286486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d292f-9238-49d9-925c-270a68aa4061 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.291058 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.291971 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.292258 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.292535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] PATH_INFO: `/v3/projects/7089fc80c56c49ceba1955b1607e6090/users/c7b35178e4a7460b8973af1ffcd05d72/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.306862 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7089fc80c56c49ceba1955b1607e6090, user_id=c7b35178e4a7460b8973af1ffcd05d72, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.309531 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.437513 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1456/2866] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:03 2022] PUT /identity/v3/projects/7089fc80c56c49ceba1955b1607e6090/users/c7b35178e4a7460b8973af1ffcd05d72/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:03.448768 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.449057 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.449321 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.477008 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] MFA Rules not processed for user `c7b35178e4a7460b8973af1ffcd05d72`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:03.478213 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:03.543700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1411/2867] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:03.548256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.611009 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76f1ea-8c7f-4926-a7bc-53a42ed94a52 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.616323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.616981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.617266 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.617543 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.618236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.620183 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.633733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:03.724792 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1457/2868] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:03 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:03.728955 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.740634 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f1c2e-8dac-4504-a529-a1792ad83117 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.799279 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.799993 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.800282 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.800565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.801346 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.803460 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.812023 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:03.943705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1412/2869] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:03 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:03.947743 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.959380 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020d17-b019-4480-9858-168cef18c2ce admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.963975 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.964629 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:03.965003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:03.965278 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:03.965997 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:03.967632 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:03.979741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1458/2870] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:03.983875 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:03.994518 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ab237-fcfc-4c28-b5b4-305874346100 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:03.999036 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:03.999941 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.000224 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.000500 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] PATH_INFO: `/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b/users/99fa39baee5a41379c16afb8f0a9f87f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.015032 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd577a2819e4daca6dfb61fbbc3805b, user_id=99fa39baee5a41379c16afb8f0a9f87f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:04.017717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:04.122864 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1413/2871] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:03 2022] PUT /identity/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b/users/99fa39baee5a41379c16afb8f0a9f87f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:38:04.126929 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:04.138904 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecac9e5-970f-4629-a54a-51ade2f3ccd2 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:04.203587 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:04.204255 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.204536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.204804 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.205547 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:04.207153 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e82f-fa82-49b2-8444-c45b7072b5ea admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:04.217553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1459/2872] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:04.221747 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:04.232632 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6b8f-67fa-47af-bc58-f42bc51a4bec admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:04.237470 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:04.238348 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.238633 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.239035 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] PATH_INFO: `/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b/users/99fa39baee5a41379c16afb8f0a9f87f/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.241329 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd577a2819e4daca6dfb61fbbc3805b, user_id=99fa39baee5a41379c16afb8f0a9f87f, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:04.244162 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:04.254872 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1414/2873] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:04 2022] PUT /identity/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b/users/99fa39baee5a41379c16afb8f0a9f87f/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:04.265995 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.266286 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.266561 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.293288 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] MFA Rules not processed for user `99fa39baee5a41379c16afb8f0a9f87f`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:04.294489 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:04.363394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1460/2874] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:04.374694 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:04.376423 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:04.438396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59e11d-bb9e-4932-a806-ef7b1e8a1e0f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:04.443564 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:04.444212 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.444492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.444770 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.453705 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:04.455222 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:04.465312 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1415/2875] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:04.553817 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:04.555307 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:04.568025 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085c35d-57a0-43e1-8280-f589e2d8cc1f None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:04.571577 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:04.572212 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:04.572491 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:04.572767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:04.633571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:04.635040 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3211f-7204-4864-bc24-fde4cba782bc service cinder] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:04.643986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1461/2876] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:08.645587 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:08.707686 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c902c-16cd-4447-b9bf-9522af6fac58 service cinder] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:08.713124 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:08.714021 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:08.714290 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:08.714570 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] PATH_INFO: `/v3/users/edc73c02c8264783913418d0e6160430` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:08.715790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc73c02c8264783913418d0e6160430)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:08.717754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:08.873071 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc73c02c8264783913418d0e6160430'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:08.873741 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc73c02c8264783913418d0e6160430'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:08.884410 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:08.886567 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc73c02c8264783913418d0e6160430'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:08.893471 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc73c02c8264783913418d0e6160430'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:08.968258 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc73c02c8264783913418d0e6160430'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:08.974597 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1416/2877] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:08 2022] DELETE /identity/v3/users/edc73c02c8264783913418d0e6160430 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:08.979954 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:08.980452 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:08.980571 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:08.987430 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=100 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:08.998605 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:08.999839 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:09.059620 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1462/2878] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:09.069781 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:09.071453 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:09.129459 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf4de7-7ee5-4237-b124-a9d63ac3a61f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:09.133555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:09.134196 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.134484 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.134761 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.143758 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:09.145195 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:09.155250 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1417/2879] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:09.551211 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:09.552894 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:09.566718 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950fd19-6781-4633-9a53-c94eb7e10016 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:09.572375 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:09.573249 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.573538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.573816 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] PATH_INFO: `/v3/projects/8fc23de0f1e6440493b295491ec3c871` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.575013 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc23de0f1e6440493b295491ec3c871)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:09.576987 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:09.677003 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc23de0f1e6440493b295491ec3c871', 'request_id': 'req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:09.683399 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc23de0f1e6440493b295491ec3c871', 'request_id': 'req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:09.766662 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc23de0f1e6440493b295491ec3c871', 'request_id': 'req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:09.778177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] The token cache is being invalidate because project 8fc23de0f1e6440493b295491ec3c871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:09.778473 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:09.779095 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:09.780665 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1463/2880] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:09 2022] DELETE /identity/v3/projects/8fc23de0f1e6440493b295491ec3c871 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:09.784831 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:09.795236 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1c8e2-200d-4bd0-bd21-9cd5c8f5504b service neutron] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:09.850249 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:09.851816 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:09.852094 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:09.852340 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] PATH_INFO: `/v3/users/c7b35178e4a7460b8973af1ffcd05d72` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:09.853553 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b35178e4a7460b8973af1ffcd05d72)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:09.855767 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:10.031201 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b35178e4a7460b8973af1ffcd05d72'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.031857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b35178e4a7460b8973af1ffcd05d72'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.044563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b35178e4a7460b8973af1ffcd05d72'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.051419 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b35178e4a7460b8973af1ffcd05d72'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.122743 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b35178e4a7460b8973af1ffcd05d72'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.125047 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:10.129455 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1418/2881] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:09 2022] DELETE /identity/v3/users/c7b35178e4a7460b8973af1ffcd05d72 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:10.530394 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:10.531853 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:10.547018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac9a9f-e0d6-4bff-82c4-354ee129fb28 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:10.606350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:10.607239 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:10.607524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:10.607798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] PATH_INFO: `/v3/projects/7089fc80c56c49ceba1955b1607e6090` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:10.608996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7089fc80c56c49ceba1955b1607e6090)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:10.611018 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:10.687409 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:10.700333 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7089fc80c56c49ceba1955b1607e6090', 'request_id': 'req-8eb6c284-e073-4097-a0f1-3293985b6e44'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.705844 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7089fc80c56c49ceba1955b1607e6090', 'request_id': 'req-8eb6c284-e073-4097-a0f1-3293985b6e44'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.766350 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7089fc80c56c49ceba1955b1607e6090', 'request_id': 'req-8eb6c284-e073-4097-a0f1-3293985b6e44'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.778870 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] The token cache is being invalidate because project 7089fc80c56c49ceba1955b1607e6090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:10.779186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.779815 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:10.781412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1464/2882] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:10 2022] DELETE /identity/v3/projects/7089fc80c56c49ceba1955b1607e6090 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:10.785334 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:10.786759 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:10.799860 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2ad89-9275-421f-a9d9-3d776ab30040 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:10.860608 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:10.861422 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:10.861704 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:10.862022 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] PATH_INFO: `/v3/users/99fa39baee5a41379c16afb8f0a9f87f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:10.863234 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99fa39baee5a41379c16afb8f0a9f87f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:10.865145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:11.038893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99fa39baee5a41379c16afb8f0a9f87f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.039560 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99fa39baee5a41379c16afb8f0a9f87f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.051429 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99fa39baee5a41379c16afb8f0a9f87f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.059107 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99fa39baee5a41379c16afb8f0a9f87f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.134180 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-45db68c8-1813-408a-9c83-7a9c91de4f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99fa39baee5a41379c16afb8f0a9f87f'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.140322 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1419/2883] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:10 2022] DELETE /identity/v3/users/99fa39baee5a41379c16afb8f0a9f87f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:11.559733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:11.561231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:11.576771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c284-e073-4097-a0f1-3293985b6e44 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:11.637590 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:11.638457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:11.638742 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:11.639022 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] PATH_INFO: `/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:11.640233 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd577a2819e4daca6dfb61fbbc3805b)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:11.642192 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:11.745808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd577a2819e4daca6dfb61fbbc3805b', 'request_id': 'req-bf46c0fd-0e45-4528-8ec2-a69c0b401689'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.751554 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd577a2819e4daca6dfb61fbbc3805b', 'request_id': 'req-bf46c0fd-0e45-4528-8ec2-a69c0b401689'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.819990 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd577a2819e4daca6dfb61fbbc3805b', 'request_id': 'req-bf46c0fd-0e45-4528-8ec2-a69c0b401689'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.831798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] The token cache is being invalidate because project 3fd577a2819e4daca6dfb61fbbc3805b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80484) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:11.832088 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.832717 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bf46c0fd-0e45-4528-8ec2-a69c0b401689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:11.834299 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1465/2884] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:11 2022] DELETE /identity/v3/projects/3fd577a2819e4daca6dfb61fbbc3805b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:11.841049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:11.841532 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:11.846599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:11.860498 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:11.861717 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:11.922604 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1420/2885] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:11.932598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:11.932598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:11.932598 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:11.950733 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:11.952011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:11.960017 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1466/2886] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:11.965145 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:11.975049 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a04e9-5b78-4f01-a896-5516b5347e95 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:11.981006 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:11.981673 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:11.981960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:11.982245 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] PATH_INFO: `/v3/domains` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:11.982989 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:38:11.983328 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:11.992248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:11.997628 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1421/2887] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:38:12.004831 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.014229 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298396e-f10c-44a2-a728-552c4643b5a0 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.018809 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.019486 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.019769 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.020045 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.020738 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.022745 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.116164 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1467/2888] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:12.120988 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.131004 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ea10f-9e58-4dd2-b42a-169a5151ebae admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.134696 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:12.188787 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.189403 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.189687 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.189967 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.190653 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.192631 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.266475 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1422/2889] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:12.267952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.278747 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e10e3-f531-4f36-9d16-6524d1ade7f2 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.283383 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.285939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.285939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.285939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.285939 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.286835 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.299010 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1468/2890] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:12.302907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.313852 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f32d1-4902-4047-a1b6-32e7bd26264f admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.318513 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.319204 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.319497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.319830 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.320565 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.322173 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eac119-d02f-4bfc-98b2-b2fe3c15dda3 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.332187 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1423/2891] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:12.336849 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.346778 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197273-f8df-40e1-b1ba-1a46340778d9 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.352142 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.353044 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.353334 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.353610 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] PATH_INFO: `/v3/projects/bd489555f895477b87e03dfb6a50e70f/users/26fc044c0b164095a8ce5b99c590c504/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.357052 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:12.369493 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd489555f895477b87e03dfb6a50e70f, user_id=26fc044c0b164095a8ce5b99c590c504, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.372210 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.483898 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1469/2892] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:12 2022] PUT /identity/v3/projects/bd489555f895477b87e03dfb6a50e70f/users/26fc044c0b164095a8ce5b99c590c504/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:12.496270 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.496635 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.496918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.525454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] MFA Rules not processed for user `26fc044c0b164095a8ce5b99c590c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:12.526661 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:12.591075 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1424/2893] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:12.595690 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.658250 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045566a-3be7-47db-b5e7-2924f5428948 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.662950 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.664696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.665090 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.665374 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.666061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.668000 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.746342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1470/2894] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:12.750344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.762092 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983063a1-0fc0-4660-ab15-9fb201408dc1 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.818205 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.818881 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.819199 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.820228 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] PATH_INFO: `/v3/users` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.820938 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.822858 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.915024 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1425/2895] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:12 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:12.918981 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.929391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef41d1-034b-4f15-bf2c-6d5a886c5d27 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.934113 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.934771 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.935056 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.935364 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.936096 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.937693 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:12.948945 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1471/2896] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:12.953000 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:12.959575 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:12.964647 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6976933-8ae6-480a-b8ae-d8af47f575bc admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:12.969232 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:12.972211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:12.972211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:12.972211 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] PATH_INFO: `/v3/projects/6dc94322d6284fb29846a108aa145295/users/9d1771a635d14601bd114422c1eee77c/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:12.986119 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc94322d6284fb29846a108aa145295, user_id=9d1771a635d14601bd114422c1eee77c, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:12.988784 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:13.094480 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1426/2897] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:12 2022] PUT /identity/v3/projects/6dc94322d6284fb29846a108aa145295/users/9d1771a635d14601bd114422c1eee77c/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:13.098401 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:13.108867 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b2518-6eb0-4c70-a0c5-97ebd8cb553b admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:13.166400 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:13.167068 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:13.167379 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:13.167652 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:13.168362 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:13.169942 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d27c3-6f47-4090-bfad-6a9c5a35bbf8 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:13.181146 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1472/2898] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:38:13.184594 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:13.195918 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d8bb8-27e0-42c7-9f4d-6f34e908b5fe admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:13.200531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:13.201413 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:13.201695 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:13.201970 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] PATH_INFO: `/v3/projects/6dc94322d6284fb29846a108aa145295/users/9d1771a635d14601bd114422c1eee77c/roles/beb4a709a14b415db991ad1980b406a9` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:13.203991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc94322d6284fb29846a108aa145295, user_id=9d1771a635d14601bd114422c1eee77c, role_id=beb4a709a14b415db991ad1980b406a9)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:13.206638 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:13.217248 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1427/2899] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:13 2022] PUT /identity/v3/projects/6dc94322d6284fb29846a108aa145295/users/9d1771a635d14601bd114422c1eee77c/roles/beb4a709a14b415db991ad1980b406a9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:13.227373 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:13.227668 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:13.227952 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:13.255075 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] MFA Rules not processed for user `9d1771a635d14601bd114422c1eee77c`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:13.256309 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:13.299528 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:13.324252 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1473/2900] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:13.334901 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:13.337304 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:13.397336 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20198-3664-4e43-bbc6-6d188604ad14 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:13.401308 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:13.401991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:13.402278 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:13.402555 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:13.411563 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:13.413007 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f04c8-cbe0-461d-934e-473c726c1d36 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:13.422396 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1428/2901] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:13.937087 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:14.000408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2704c-16df-461b-85c3-f016d73c8b09 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:14.005061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:14.005936 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:14.006223 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:14.006500 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] PATH_INFO: `/v3/users/26fc044c0b164095a8ce5b99c590c504` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:14.007720 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fc044c0b164095a8ce5b99c590c504)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:14.009760 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:14.186098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fc044c0b164095a8ce5b99c590c504'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.193941 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fc044c0b164095a8ce5b99c590c504'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.206538 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fc044c0b164095a8ce5b99c590c504'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.207274 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fc044c0b164095a8ce5b99c590c504'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.269154 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fc044c0b164095a8ce5b99c590c504'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.276318 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1474/2902] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:13 2022] DELETE /identity/v3/users/26fc044c0b164095a8ce5b99c590c504 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:14.280908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:14.281389 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:14.281497 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:14.298626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:14.299870 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:14.358866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1429/2903] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:14.368741 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:14.370149 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:14.428061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c60b3-cc95-43d1-bdad-7d089be427ca admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:14.432388 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:14.432974 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:14.433256 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:14.433573 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:14.442582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:14.444043 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:14.454413 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1475/2904] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:14.859551 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:14.861087 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:14.874080 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f501c2a-ff6e-4638-88e3-4a6c6916bbe0 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:14.878889 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:14.879943 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:14.880229 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:14.880663 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] PATH_INFO: `/v3/projects/bd489555f895477b87e03dfb6a50e70f` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:14.881932 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd489555f895477b87e03dfb6a50e70f)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:14.884568 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:14.980960 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd489555f895477b87e03dfb6a50e70f', 'request_id': 'req-339b2f22-0932-441d-a9c5-5f893a9e43c1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.986412 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd489555f895477b87e03dfb6a50e70f', 'request_id': 'req-339b2f22-0932-441d-a9c5-5f893a9e43c1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.993155 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd489555f895477b87e03dfb6a50e70f', 'request_id': 'req-339b2f22-0932-441d-a9c5-5f893a9e43c1'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:14.995733 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:15.064505 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] The token cache is being invalidate because project bd489555f895477b87e03dfb6a50e70f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:15.064790 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.065516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.067083 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1430/2905] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:14 2022] DELETE /identity/v3/projects/bd489555f895477b87e03dfb6a50e70f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:15.071680 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:15.076508 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:15.081756 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86508737-b01e-4b92-9a1e-67ec909b7fd7 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:15.137553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:15.138535 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:15.138821 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:15.139100 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] PATH_INFO: `/v3/users/9d1771a635d14601bd114422c1eee77c` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:15.140359 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1771a635d14601bd114422c1eee77c)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:15.143672 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:15.324398 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1771a635d14601bd114422c1eee77c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.331061 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1771a635d14601bd114422c1eee77c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.343437 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1771a635d14601bd114422c1eee77c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.344115 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1771a635d14601bd114422c1eee77c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.408387 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-bbcd734e-88d0-4fb2-bdb5-b99f4d88c23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1771a635d14601bd114422c1eee77c'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:15.415786 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1476/2906] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:38:15 2022] DELETE /identity/v3/users/9d1771a635d14601bd114422c1eee77c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:38:15.827510 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:15.828961 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:15.843520 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2f22-0932-441d-a9c5-5f893a9e43c1 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:15.901244 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:15.902034 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:15.902320 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:15.902599 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] PATH_INFO: `/v3/projects/6dc94322d6284fb29846a108aa145295` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:15.903804 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc94322d6284fb29846a108aa145295)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:15.905727 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.009391 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc94322d6284fb29846a108aa145295', 'request_id': 'req-6576a8d4-88bf-4896-bfb6-196ab9e6585c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:16.014775 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc94322d6284fb29846a108aa145295', 'request_id': 'req-6576a8d4-88bf-4896-bfb6-196ab9e6585c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:16.020866 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc94322d6284fb29846a108aa145295', 'request_id': 'req-6576a8d4-88bf-4896-bfb6-196ab9e6585c'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:16.070891 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] The token cache is being invalidate because project 6dc94322d6284fb29846a108aa145295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:38:16.071408 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:16.076987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-6576a8d4-88bf-4896-bfb6-196ab9e6585c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:38:16.076987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1431/2907] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:38:15 2022] DELETE /identity/v3/projects/6dc94322d6284fb29846a108aa145295 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:16.079846 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.080323 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.080436 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.099607 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:16.100800 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:16.160171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1477/2908] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:16.168192 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.168477 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.168754 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.187908 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:16.189116 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:16.197856 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1432/2909] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:16.201986 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.211825 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e6684-ab55-40cf-b43b-6a748910a12d None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.216319 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.216968 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.217267 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.217524 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.218241 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80484) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:38:16.218546 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.220167 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.232130 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1478/2910] 10.222.0.59 () {58 vars in 1152 bytes} [Thu Jul 14 11:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:38:16.236363 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.246177 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890a18-51d3-4fb4-af41-fec74c3a6d00 None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.250854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.251531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.251817 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.252095 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] PATH_INFO: `/v3/projects` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.252777 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.254665 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.338671 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1433/2911] 10.222.0.59 () {60 vars in 1152 bytes} [Thu Jul 14 11:38:16 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:16.342852 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.353582 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a12ad-de07-4260-921c-775ca83548fe admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.358585 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:16.410980 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.411608 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.411891 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.412166 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] PATH_INFO: `/v3/users` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.412907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.415457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.511340 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1479/2912] 10.222.0.59 () {60 vars in 1143 bytes} [Thu Jul 14 11:38:16 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:38:16.515447 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.526558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d891adc-3bab-4bf1-861e-51378f9bd06d admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.531193 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.531857 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.532172 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.532454 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.533241 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.534893 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.546018 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1434/2913] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:38:16.549957 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.561405 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15daf-b31a-4978-a0ce-929351e16238 admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.566702 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.567391 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.567682 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.567966 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.568696 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.570412 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658ab35-d919-452f-82ed-65063bb81ccc admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.580442 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1480/2914] 10.222.0.59 () {58 vars in 1121 bytes} [Thu Jul 14 11:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:38:16.584572 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.599253 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d052a5-fab1-4db9-95f8-63f704f36e62 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.604957 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.605848 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.606136 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.606411 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] PATH_INFO: `/v3/projects/c756dc2da35a4b5788a3303297f63fcc/users/c87613bba9ef4732b831b00ff937bc54/roles/f64ed86228d24c5b8f70e7a0e8625010` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.622797 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c756dc2da35a4b5788a3303297f63fcc, user_id=c87613bba9ef4732b831b00ff937bc54, role_id=f64ed86228d24c5b8f70e7a0e8625010)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.625516 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.694991 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1435/2915] 10.222.0.59 () {60 vars in 1482 bytes} [Thu Jul 14 11:38:16 2022] PUT /identity/v3/projects/c756dc2da35a4b5788a3303297f63fcc/users/c87613bba9ef4732b831b00ff937bc54/roles/f64ed86228d24c5b8f70e7a0e8625010 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:38:16.706050 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] REQUEST_METHOD: `POST` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.706342 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.706651 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.735408 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.auth.core [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] MFA Rules not processed for user `c87613bba9ef4732b831b00ff937bc54`. Rule list: `[]` (Enabled: `True`). {{(pid=80484) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:38:16.740231 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:16.815011 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1481/2916] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:38:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:38:16.826966 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:38:16.828711 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:38:16.850986 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:16.890378 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f9f7-ca12-43d1-a337-c5248f319082 admin admin] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:38:16.895462 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86f8e914f3f4312a60791af443b2f5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:38:16.896499 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:38:16.897053 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:38:16.897624 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:38:16.909147 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:38:16.912596 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e60d-98b0-4058-861b-9e95f55a07c6 service cinder] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:38:16.920700 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1436/2917] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:38:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:38:26.811236 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:38:26.961697 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=17 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:39:02.644803 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:39:02.710798 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fbf60-62e4-41a9-a861-56cc1eb72486 None None] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:39:02.715177 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:39:02.716042 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:39:02.716324 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:39:02.716599 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] PATH_INFO: `/v3/users/c87613bba9ef4732b831b00ff937bc54` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:39:02.717767 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87613bba9ef4732b831b00ff937bc54)` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:39:02.720296 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:39:02.895848 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87613bba9ef4732b831b00ff937bc54'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:02.902907 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87613bba9ef4732b831b00ff937bc54'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:02.916463 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87613bba9ef4732b831b00ff937bc54'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:02.917217 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87613bba9ef4732b831b00ff937bc54'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:02.992251 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.notifications [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87613bba9ef4732b831b00ff937bc54'} {{(pid=80484) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:02.998653 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1482/2918] 10.222.0.59 () {58 vars in 1223 bytes} [Thu Jul 14 11:39:02 2022] DELETE /identity/v3/users/c87613bba9ef4732b831b00ff937bc54 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:39:03.003365 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] REQUEST_METHOD: `POST` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:39:03.003878 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:39:03.003987 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:39:03.022273 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.auth.core [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] MFA Rules not processed for user `23fec8c4e37c41799ff49ae249e16e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:39:03.023531 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:39:03.084372 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1437/2919] 10.222.0.59 () {58 vars in 957 bytes} [Thu Jul 14 11:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:39:03.093934 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:39:03.096270 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:39:03.153457 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f19fb4-f6cf-4bff-ad49-218beab2b6ee admin admin] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:39:03.157080 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a463c3c3f974bae83a6f4b21729f503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:39:03.157992 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:39:03.158277 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:39:03.158556 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:39:03.167576 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:39:03.168996 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:39:03.178715 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1483/2920] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:39:03.571268 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] Authenticating user token {{(pid=80483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:39:03.572907 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.fernet_utils [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:39:03.585764 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06811d57-384b-43d5-b03e-98acfad4b1ab None None] Validating token access rules against request {{(pid=80483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:39:03.590434 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23fec8c4e37c41799ff49ae249e16e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7da05e4cf2b444729a9cbe3b6d5db535', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:39:03.591344 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:39:03.591626 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:39:03.591900 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] PATH_INFO: `/v3/projects/c756dc2da35a4b5788a3303297f63fcc` {{(pid=80483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:39:03.593084 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c756dc2da35a4b5788a3303297f63fcc)` {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:39:03.595815 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] RBAC: Authorization granted {{(pid=80483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:39:03.725558 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c756dc2da35a4b5788a3303297f63fcc', 'request_id': 'req-481feabc-8f3f-44cc-bcce-93a16f93c8b9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:03.731492 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c756dc2da35a4b5788a3303297f63fcc', 'request_id': 'req-481feabc-8f3f-44cc-bcce-93a16f93c8b9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:03.737854 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c756dc2da35a4b5788a3303297f63fcc', 'request_id': 'req-481feabc-8f3f-44cc-bcce-93a16f93c8b9'} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:03.809130 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] The token cache is being invalidate because project c756dc2da35a4b5788a3303297f63fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:39:03.809414 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:03.810025 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG keystone.notifications [None req-481feabc-8f3f-44cc-bcce-93a16f93c8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:39:03.811584 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: [pid: 80483|app: 0|req: 1438/2921] 10.222.0.59 () {58 vars in 1232 bytes} [Thu Jul 14 11:39:03 2022] DELETE /identity/v3/projects/c756dc2da35a4b5788a3303297f63fcc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:39:13.174765 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:39:13.805904 dv-cinder-822821-5-iscsi devstack@keystone.service[80483]: DEBUG dbcounter [-] [80483] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:40:24.459541 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] Authenticating user token {{(pid=80484) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:40:24.461186 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:24.531071 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e5620-bc53-4942-87b5-18a60e583419 service neutron] Validating token access rules against request {{(pid=80484) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:40:24.591136 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf80b4722094475934134be619652e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86229daf0943484393a30086a3139985', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80484) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:40:24.591808 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] REQUEST_METHOD: `GET` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:40:24.592098 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] SCRIPT_NAME: `/identity` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:40:24.592384 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80484) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:40:24.594171 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.fernet_utils [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80484) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:40:24.605536 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:40:24.607396 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7fd3f-9427-43e5-b2a4-41aa07102bdb service placement] RBAC: Authorization granted {{(pid=80484) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:40:24.617893 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: [pid: 80484|app: 0|req: 1484/2922] 10.222.0.59 () {60 vars in 1365 bytes} [Thu Jul 14 11:40:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:40:34.613553 dv-cinder-822821-5-iscsi devstack@keystone.service[80484]: DEBUG dbcounter [-] [80484] Writing DB stats keystone:SELECT=58 {{(pid=80484) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}